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ABSTRACT 

The reliance upon interorganizational relationships in health and public health 
systems is extensive; however, there is a shortage of evidence that focuses on stakeholder 
network performance. Responding to this gap and the call for a systems approach, the 
study's first research question asks whether community based stakeholder networks 
operating within the same context exhibit commonalities in network-level attributes and 
characteristics relative to their performance. The second question focuses on 
organizational level variables, asking: do the relative connectivity and structural positions 
of member organizations affect stakeholder network performance? 

In a mixed method, cross case research design, the characteristics and attributes of 
five stakeholder networks were analyzed and compared. Whole-network, dyad and 
organizational measures were examined through network analysis, and analyzed with 
performance measures based on external stakeholder assessments, network self 
assessments and local early intervention assessments. Emergent trends were then 
analyzed through content analysis. Focusing on the dynamic interactions of public health 
agencies with public and private organizations that affect health, the five stakeholder 
groups are goal-directed networks working to ensure standardized developmental 
screening, referral and follow through for children in their Colorado communities. The 
stakeholder networks were mobilized to overcome the segmented system of early 
intervention services. 
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The study found that there were no relationships in whole-network level attributes 
and characteristics relative to performance. Investigation into the second research 
question revealed a positive relationship between network structures with a system- 
building organization in a key position relative to network performance; and a negative 
relationship between network structures with a service provider organization in a key 
position relative to network performance. There was also a positive relationship between 
the levels of integration of primary care providers relative to network performance. 
Results also showed that trust among organizations within a stakeholder network is not a 
sufficient condition for network performance. 

The study findings provide new evidence about how different characteristics and 
attributes of stakeholder networks affect performance, adding to a much needed 
knowledge base to guide the use, formation and management of organizational networks 
for strong health and public health systems. 

The form and content of this abstract are approved. I recommend its publication. 

Approved: Christine Martell 
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CHAPTER I 

INTRODUCTION 

Background 

America aims to be "a society in which all people live long, healthy lives" (United 

States Department of Health and Human Services [HHS], 2012, Message from the 

Secretary, para. 1). The goal's wording is deceptively short and simple; the path towards 

its achievement is anything but. A healthy society is vitally important to American life: 

health is a necessity at the individual level for quality of life, at the community level for 

thriving cities and productive workforces, and at the national level for a growing 

economy and prosperous citizenry. The problem is that America spends far more than 

any industrialized nation in the world, yet ranks only mediocre in most important health 

outcome measures, as detailed in Table 1. 1 below. Ninety-seven percent of total health 

care costs are related to medical care provided on an individual basis (Institute of 

Medicine [IOM], 2012a). With costs continuing to increase, there is a consensus among 

health care and policy leaders that reliance on the primary care model has become fiscally 

unsustainable for the nation (IOM, 2012a). Further magnifying the challenges of a 

healthy society is the fact that barriers to health are multifaceted, extensive and 

interrelated. 

Table 1.1 U.S. Health Rankings. 

US Ranking out of total OECD Countries 

Life Expectancy Infant Mortality Maternal Mortality 

26th of 34 30th of 34 (2008 data) 25th of 34 (2007 data) 

Source: Organization for Economic Co-operation and Development [OECD], 2009. 



Health Care and Public Health Systems 

Ideas to improve the health care system abound, but what has become clear to 
researchers and policy makers alike is that a systems approach is required. Health care in 
America is a complex and multifarious set of systems, with layers of subsystems. There 
are primary care and specialist physicians providing care to individuals, focused for the 
most part, on treating existing medical symptoms and conditions. A bevy of insurance 
companies, medical equipment and supply firms, pharmaceutical companies and billing 
administrators are heavily invested in the provision of care, with their own biases and 
objectives. Hospitals, some for-profit, some nonprofit, are part of the system's 
infrastructure, along with government regulators, medical universities and other health 
training facilities. Research and development, from both the private and public sectors, 
continue to change the trajectories of the health care business models, clinical services, 
future research priorities and medical education. 

Over $2.6 trillion is spent each year on medical care in the health care system 
described above, and yet that does not represent the whole picture (IOM, 2012b). 
Ensuring a healthy nation clearly requires a system of health care that includes broad 
access to providers and the provision of quality care, but it also requires efforts to prevent 
disease and injury; efforts to promote healthy lifestyles; and the assurance of safe 
conditions and environments in which to live and work (IOM, 2012b). Thus, there is the 
additional layer of the public health system, which is focused on preventing health 
conditions such as chronic disease, cancer, disability and injury, at the population level 
(Detels, Beaglehole, Lansang, & Gulliford, 2011). Public health further increases the 



complexity of health in America, bringing a whole new set of actors, funding streams, 
objectives and level of focus. 

The public health system in America is anchored by governmental public health 
organizations, specifically, the 5 1 state public health agencies, 2,794 local public health 
agencies, and 565 American Indian and Alaska Native tribal public health agencies 
(National Association of County and City Health Officials [NACHO], 2010). These 
organizations are charged with the task of providing 10 essential public health services 
(EPHS), detailed in Key Terms and Definitions in this Chapter. However, these 
governmental organizations do not have the capacity or funding to be able to provide the 
EPHS alone. To fulfill their charge, they partner with organizations within all sectors, 
serving as a "catalyst for engaging multiple stakeholders to confront community health 
problems" (IOM, 2012a, p. 1-9.) Partners at the national level include federal agencies 
such as The Health Resources and Services Administration (HRSA), Centers for Disease 
Control (CDC), and National Institutes of Health (NIH); and national nonprofit 
organizations such as the Institute of Medicine (IOM) and Robert Wood Johnson 
Foundation. Partners at the state and community level include primary care providers, 
universities and hospital systems, federally qualified health centers, community based 
health centers, and mental health centers, among others. These partners highlight the 
interwoven complexity and interdependency of the health care delivery system with the 
public health system. Additional state and local partners may include organizations 
outside the realm of primary care, such as business groups and faith based organizations. 
Community participation is also vital to public health, and thus, local partners also often 



include advisory boards, nonprofit and grassroots organizations, as well as families and 
individuals (IOM, 2012a). 

Traditionally perceived to be two separate systems, there is an increasing call for 
primary care and public health to integrate because many organizations work within both 
primary care and public health, sharing the common goal of improving health. The 
integration of these two systems is seen as the crucial element to "enhance the capacity of 
both sectors to carry out their respective missions and link with other stakeholders to 
catalyze a collaborative, intersectoral movement toward improved population health" 
(IOM, 2012a, p. S-l). It is hoped that better integration will enable both primary care and 
public health to capitalize on their synergies, and better achieve an overall improvement 
in heath together. 

Statement of Research Problem 

Given the complexity of tasks in assuring conditions for health, and the breadth of 
organizations that are involved, it has been recognized by health leaders and policy 
makers across the country that change at the system level is required to enable substantial 
and sustainable improvements in the health of Americans (IOM, 2012a). A systems 
approach, and an understanding of system level factors (and their impact on the delivery 
of health care and public health services) is thus critical. Interorganizational relationships 
make up an important system level factor: partnerships and collaboration are an integral 
component of the public health system's organization and structure; an increasingly 
common method of health and public health service delivery; and an important influence 
for population health outcomes as an essential public health service unto themselves 
(IOM, 2011). 



Interorganizational Relationships 

Interorganizational relationships are a core feature of the health and public 
system's organization and structure for the fundamental reason that assuring conditions 
for population health is beyond the scope and abilities of any one organization or 
government agency (NACHO, 2012). Stakeholder groups and partnerships are the 
foundation of the local public health infrastructure, and can include any individual, 
group, or organization that affects, or can be affected by, actions to improve population 
health. Through "a range of relationships that foster the sharing of resources and 
accountability for health improvement" (National Public Health Performance Standards 
Program [NPHPSP], 2007, p. 24), these interorganizational relationships enable public 
health agencies to carry out their core functions and essential services. 

Interorganizational relationships are clearly important in the health and public 
health systems where public, private and nonprofit organizations work together to assure 
the conditions for health; however, there is a significant lack of evidence in this area. 
The Literature Review in Chapter III chronicles the lack of robust research focused on 
effective and efficient interorganizational relationships in health and public health 
systems. Further, the 2012 Public Health Services and Systems national research agenda, 
created by research scientists and health and public health practitioners and leaders, has 
identified research that focuses on interorganizational relationships in public health as an 
important research priority (Consortium, 2012). The systems approach enables research 
that emphasizes these important interrelations between and among the various actors 
working in health care and prevention. 



Theoretical Framework 

In the field of systems theory, a system is defined as "a set of elements standing in 
interrelation among themselves and with the environment" (National Cancer Institute 
[NCI], 2007, p. 159). Thus, a systems approach requires investigation of the whole, 
rather than just an actor or aspect within a system (Mabry, 201 1; Mabry, Olster, Morgan, 
& Abrams, 2008; Gillies, 1982). 

Fundamentally, systems theory emphasizes that a system consists of a set of 
actors and their relationships. Thus when the US health care and public health systems 
are viewed through the lens of system theory, the relationships between and among the 
many actors described above are as critical to the system as the actors to which they 
belong. These relationships can have tremendous impact on the performance of the 
system. To this end, there are significant opportunities to maximize and utilize these 
relationships to leverage the resources, knowledge, information, and services to improve 
the system, and ultimately, help to support a nation in which all people can live healthier 
and more productive lives (United States Department of Health and Human Services 
[HHS], 2012). The idea that a system is greater than the sum of its parts is at the heart of 
this approach. 

A systems framework, which builds upon system theory's holistic perspective and 
assumptions of interdependencies, is thus a strong research approach to investigate 
interorganizational relationships embedded in the health and public health systems. Such 
an approach is further strengthened by theories of network structure, which aim to foster 
an understanding of effective relationships among actors and stakeholders to improve 
collaboration, information and resource flows and exchanges, and to help reduce 



redundancies (Kilduff & Tsai, 2003). A novel methodology in a systems approach is 
network analysis, which holds the potential for generating insight and facilitating 
strategic management of relationships between actors and stakeholders within a system. 
Many theories of network behavior underpin this type of systems approach, and the core 
network concepts, as detailed in the methodology in Chapter IV, have broad acceptance 
in systems research (NCI, 2007). The use of network analysis as a methodology enables 
exploration of the interorganizational relationships that are so important to the complex 
health and public health systems, and enables investigation of unanswered research 
questions, such as those in this study. 

Research Questions 

Embracing a systems approach, and responding to both the PHSSR research 
agenda (Scutchfield, Perez, Monroe & Howard, 2012) and the call to bridge primary care 
and public health (IOM, 2012a), this study asks: Do community based stakeholder 
networks operating within the same context exhibit commonalities in network-level 
attributes and characteristics relative to their performance? The second research 
question focuses upon organizational level variables, asking: Do the relative connectivity 
and structural positions of member organizations affect stakeholder network 
performance? 
Study Design 

In a mixed method, cross case research design, this study investigates five 
stakeholder networks working in the context of early childhood intervention (EI), a 
subsystem that spans the health and public health systems. The EI system at the national, 
state and local levels is described in Chapter II. 



The relational and network measures of the networks will be examined through 
social network analysis (SNA), and analyzed with performance measures based on 
external assessments, network self assessments, and early intervention services data. 

Population: Focusing on the dynamic interactions of primary care providers and 
public health agencies with other community based public and private organizations, the 
population of this study includes five stakeholder networks working to ensure 
standardized developmental screening, referral and follow up for children in their local 
community in Colorado. These networks included stakeholders such as the local public 
health agency, early intervention service provider, early childhood council and providers, 
primary care providers, school districts and other organizations in the community with an 
interest or concern with the local system of early intervention. These networks were 
mobilized to overcome the segmented system of identification and delivery for early 
intervention services. 

Research Setting: Early intervention is a system of coordinated services that 
promotes the growth and development of children during the critical early years of brain 
development. The goal of early intervention is to assure that families who have children 
with disabilities or developmental delays, ages birth to three, receive resources and 
supports that assist them in maximizing their child's development. Research has shown 
that skills and abilities can be fostered and developed during those early years of brain 
development that may be unattainable if the learning process starts later in life 
(Mackrides & Ryherd, 201 1). 

Utilizing systems theory as the general framework, and network theories to 
examine specific measures, this investigation focuses on the interorganizational 
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relationships that are inherent in each stakeholder network within the EI system that 
spans the health care delivery system, the public health system, as well as human services 
and education systems. This study investigates how different network attributes and 
characteristics at the network level (referred to hereafter as whole-network) and the 
individual organization/dyad level (referred to hereafter as organizational level) including 
structure, connectivity, trust and value affect performance. Content analysis is used to 
further investigate patterns that emerge from the network analysis. 

Theoretical background: While systems theory focuses this investigation on the 
interrelation among and between the multitude of organizations that are a part of the 
health and public health systems to ensure children's healthy development; network 
theories provide an important frame of reference in their assertions that relationships are 
not only productive resources, but that well structured networks can be more effective 
than large inclusive networks (Moynihan, Pro van & Lemaire, 2012; Burt, 1997). This has 
a significant application in public health practice, where state and local public health 
system performance is typically only assessed, in part, by counting the number of 
stakeholders in partnerships (NPHPSP, 2007). Provan, Fish, and Sydow's (2007) 
typology of organizational network research provide a framework for analysis, while 
Varda, Chandra, Stern & Lurie's (2008) core dimensions of connectivity; Karckhardt's 
(1992, 2010) strength of strong ties findings; Granovetter's (1973, 1992) strength of weak 
ties theory; and Burt's (1995, 2000, 2001) brokerage research inform the 
operationalization of network variables to analyze and explore patterns at the whole- 
network and organization level across the five stakeholder networks. 



Combining the quantitative methodology of social network analysis to measure 
the social structure, with the exploratory nature of qualitative inquiry to explore emergent 
patterns, this study aims to provide insight into the dimensions of partnership process and 
performance. The study's findings provide new empirical evidence about how different 
characteristics and attributes of partnership affect performance, adding to a much needed 
knowledge base to guide the use, formation and management of interorganizational 
relationships for effective health and public health systems. 
Plan for Dissertation 

This investigation is presented in six chapters, with the aim to provide insight into 
the attributes and characteristics of stakeholder networks (working to improve the system 
of early intervention), and their impact on network performance. Chapter II explores the 
policy context of EI nationally, and describes the segmented system of EI. The chapter 
describes how the EI system is implemented at the state and local levels in Colorado. To 
this end, it introduces the multiple legislated agencies, along with their respective goals, 
from public health, human services and education, and describes the reasons behind the 
call for greater integration of primary care providers. The chapter also details the specific 
types of organizations that are working together in the five stakeholder networks in this 
study. 

Chapter III begins with the literature of systems theory, and offers an application 
in the context of early intervention. The chapter then navigates the complex body of 
social and organizational network theories. Demonstrating a lack of empirical evidence 
that relates whole-network attributes to network performance, the exploratory approach 
to the first research question is explained. The chapter then synthesizes the network 
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theories that inform the investigation into the second research question, as well as offers a 
discussion on the methodologies and the assumptions contained within. 

Chapter IV Offers a detailed description of all aspects of the research design and 
methods used in the study, including network analysis to measure the network attributes 
and characteristics, and content analysis to explore the emergent findings from the cross 
case analysis. The chapter describes the mixed-method paradigm utilized in this study, as 
well as the specific approach and procedures for data collection. It also includes an 
explanation of the epistemo logical grounding for the methods and addresses how they are 
a strong fit for this line of inquiry. The chapter then offers a detailed description of the 
dimensions of network performance, the dependant variable in this study, and the whole- 
network variables and organizational level variables, the independent variables in this 
study. The chapter concludes with the limitations of the study, along with the reliability 
and validity of the approach. 

Chapter V details the findings of the network analysis and identifies patterns that 
exist across the five cases. The chapter is organized around the research questions, first 
presenting results from the cross-case analysis of the whole-network variables; and then 
presenting results from the cross-case analysis of the attributes and characteristics at the 
organizational level. The chapter concludes with findings from the content analysis, 
which was undertaken to better understand a distinction in organizational type that 
emerged from the cross case analysis. 

Chapter VI concludes the study with a discussion on the research findings. The 
first part of this discussion is organized around the research questions, and offers a 
summary of the findings, interpretation of the data, and conclusions drawn from the 

11 



information. A discussion about the contribution of the research to organizational 
network theory is offered, as well as the implications for health and public health 
systems. Four conclusions summarize the contributions to both knowledge and practice. 
The chapter ends with recommendations for future research. 
Key Definitions and Terms 

System: A system is a collection of independent elements that are interrelated 
among themselves and within their environment, and organized in a meaningful way to 
accomplish an overall goal (Hayajneh, 2007). 

Systems approach: An approach to research or analysis that uses systems theory 
and methods in an organized framework to investigate systems. The National Cancer 
Institute's Initiative on the Study and Implementation of Systems (NCI, 2007) 
synthesizes the four key systems approaches as: 

1. "Systems organizing to understand and foster the development of participatory, 
complex, and adaptive collaborative systems; ensure their effective facilitation 
and management; and encourage productive system action and learning. 

2. System dynamics to understand and model the complex dynamic interactions 
involved in the system. 

3. System networks to understand and analyze effective collaborative relationships 
among stakeholders, improve collaboration strategies, and help reduce duplication 
of effort. 

4. Systems knowledge to develop and manage the knowledge infrastructure 
required for effective dissemination and evolution of scientifically credible, 
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evidence-based practices, together with an effective strategy to package, deliver, 

and maintain this knowledge" (p. 2). 
Systems methods are tools, techniques or procedures to enable investigation of systems. 
Network analysis, system dynamics modeling, and structured conceptualization are all 
systems methods (NCI, 2007). 

Primary Care: Primary care in America is defined by the Institute of Medicine 
(IOM) (1996, p. 1) as the provision of "integrated, accessible health care services by 
clinicians who are accountable for addressing a large majority of personal health care 
needs, developing a sustained partnership with patients, and practicing in the context of 
family and community." Most primary care is delivered through the private sector, but it 
can also be delivered through government agencies, including the Veterans Health 
Administration (VHA) hospitals and Health Resources and Services Administration 
(HRSA)'s safety net clinics. Regardless of whether it is private or public, primary care 
physicians often provide a point of first contact for health related issues, and aim to offer 
comprehensive and coordinated care (Starfield & Horder, 2007). The medical home 
model of primary care encompasses the characteristics of care that are considered 
essential for all children: "accessible, continuous, comprehensive, family centered, 
coordinated, compassionate, and culturally effective" (Strickland, Jones, Ghandour, 
Kogan, & Newacheck, 201 1, p. 605). 

Public Health: The IOM defines public health's fundamental charge as "fulfilling 
society's interest in assuring conditions in which people can be healthy" (IOM, 1988, p. 
140). Public health encompasses a diverse group of public and private stakeholders 



13 



(including the health care delivery system) working in a variety of ways to contribute to 
the health of society. 

Essential Public Health Services: The ten Essential Public Health Services 
(EPHS) provide a more specific definition of public health and a framework for the local 
public health systems. Each of the states' public health departments and local health 
agencies are charged with carrying out the EPHS. The Center for Disease Control and 
Prevention (2012) details the EPHS: 

1 . "Monitor health status to identify and solve community health problems. 

2. Diagnose and investigate health problems and health hazards in the 
community. 

3. Inform, educate, and empower people about health issues. 

4. Mobilize community partnerships and action to identify and solve health 
problems. 

5. Develop policies and plans that support individual and community health 
efforts. 

6. Enforce laws and regulations that protect health and ensure safety. 

7. Link people to needed personal health services and assure the provision of 
health care when otherwise unavailable. 

8. Assure competent public and personal health care workforce. 

9. Evaluate effectiveness, accessibility, and quality of personal and population- 
based health services. 

10. Research for new insights and innovative solutions to health problems" (para. 
3). 
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Significance 

Building a health care system that meets the needs of all Americans is an 
ambitious goal for the government, and requires both comprehensive, well-informed 
policy, as well as successful implementation through effective systems. 
Interorganizational relationships are fundamental in health and public health systems 
where public, private and nonprofit organizations work together to assure the conditions 
for population health. Yet given the significant reliance on these relationships, there is 
still much to learn. Uncertainty abounds about when organizational networks are an 
effective or efficient choice, and how to build and foster effective and efficient 
interorganizational relationships. It is important that health practitioners and leaders are 
armed with the knowledge to prompt strategic and thoughtful discussion to foster 
stronger inclusion, involvement and commitment where it most makes sense and will 
have the greatest impact. An understanding of interorganizational relationships is thus 
critical in this endeavor, and has twice been highlighted as an area of needed research in 
the last decade (Scutchfield et al, 2012). 

The gap in knowledge may exist because traditional research models are 
challenged by the large and complicated array of actors and variables within health and 
public health systems, and are often not able to take into account that fact that the 
systems are comprised of interrelated actors, and are interrelated themselves. Further, the 
systems are dynamic: they are always changing as feedback from one element of the 
system changes the course for another. A majority of the research into health care 
delivery in the twentieth century has been through reductionism, "the process of 
attempting to understand a problem by first deciphering its components" (NCI, 2007, p. 
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13; Barabasi, 2002). These traditional approaches have left important research questions 
unanswered and tremendous gaps in our knowledge of how to improve the health and 
public health systems in America. Given this, a systems approach, which guides this 
study, has gained more attention and traction in the research and practitioner arenas as a 
way to examine and explore the interrelated elements of the systems, and ultimately learn 
how to better improve and support the health of Americans. 

This investigation aims to provide stakeholders the knowledge to begin 
positioning their stakeholder networks to most effectively achieve their goals and 
positively influence health outcomes, while maximizing the return on their investment 
within the networks. Ultimately, stakeholder networks need to be effective in achieving 
their goals in order to be useful as a tool for policy implementation, and the better we 
understand the dimensions of interorganizational relationships that affect their success, 
and how, the more equipped we are to harness and leverage the strengths of these 
relationships to improve health outcomes in America. 
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CHAPTER II 
RESEARCH CONTEXT: SYSTEM OF EARLY INTERVETION 

Background 

America's aim to be a country where all citizens live long, healthy lives is a 
considerable challenge, with a complex set of systems in place to carry out the mission 
(HHS, 2012). Inherent in such an endeavor is the important goal of meeting children's 
health needs, especially for those children who are born with disabilities or those who 
experience delays in development during their early childhood. The literature shows that 
12-16 percent of children in America have at least one developmental delay (Mackrides 
& Ryherd, 201 1). Early intervention (EI) systems exist to provide coordinated supports 
necessary for these children to reach their full potential. EI is a system of coordinated 
services that promotes the child's growth and development and supports families during 
the critical early years of early brain development. The EI system is especially complex, 
given that it is legislated at the federal level but implemented at the state and local levels. 
Further, the EI system extends across the public health system, systems of education 
and/or human services, depending upon the state, all the while being called to integrate 
with part of the health care delivery system. Set at the nexus of primary care and public 
health integration, this study is focused on this multifaceted subsystem for childhood 
development. 
Early Intervention Services 

The purpose of EI is to ensure that families who have children with disabilities or 
developmental delays, ages birth to three, receive resources and support that assist them 
in maximizing their child's development. Early intervention services are focused on the 
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development that babies typically experience during the first three years of life, 
including: cognitive development (such as thinking, learning, solving problems); physical 
skills (reaching, rolling, crawling, and walking); communication; and social development 
(National Dissemination Center for People with Disabilities [NICHY], 2012). 

Early intervention services provide vital support so that children with 
developmental needs can thrive and grow. Services are often provided by physical 
therapists (PT), occupational therapists (OT), and speech and language pathologists, 
audiologists, psychologists, early childhood educators or nurses. These therapists work 
with the children to develop neural pathways, muscles, skills, and knowledge that are not 
developing in a typical manner, and provide the foundation for language, reasoning, 
problem solving, behavior, and emotional health. Therapists also provide support services 
to the families such as family education and counseling, parent support groups, special 
instruction, or assistive technology devices and services (NICHY, 2012). 

There is a substantial body of research that demonstrates the power and necessity 
of providing intervention services to children with disabilities and developmental delays 
at young ages. Research has shown that early intervention services may: 

• Improve children's gains in developmental, social, and educational area; 

• Reduce future costs of services and health care needs (such as special education, 
therapies or rehabilitation) for both families and government systems; 

• Reduce stress and feelings of frustration for children and families; 

• Help children to reach their maximum potential and become productive, 
independent individuals (Bright Tots, 2012). 
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Although there is still much to be learned about the services that work best for different 
children and families, "the combined body of existing research clearly underscored the 
need to intervene early to enable children with delays and disabilities to reach their full 
potential" (Hebbeler, Spiker, Bailey, Scarborough, Mallik, Simeonsson, et al, 2007, p. 1- 

1)- 

The benefits of EI services are felt beyond the children and families who receive 
them. Research also shows that such services can reduce the prevalence of 
developmental and behavioral disorders as children get older. Further, researchers Hess 
and Van Landeghem (2005, p. 3) have found that early intervention efforts for children 
and their families are not only beneficial and cost-effective to the health system, but can 
also reduce the prevalence or necessity "for more costly interventions and outcomes such 
as welfare dependency and juvenile detention". If children do not receive needed 
services and support, they are not only less likely to reach their developmental potential; 
there may also be long-term consequences for the health, education, child welfare, and 
justice systems. 

The importance of early intervention services for infants and toddlers with 
disabilities is well documented in the literature (Jones & Ziglar 2002; Haskins, 1989); 
that is not the investigation here. When children receive EI services, there seems to be 
little doubt in the research of the positive benefits for the child directly, as well as 
indirectly to government funded health, education, child welfare, and juvenile justice 
systems. The problem is that the implementation of the policy is through a segmented 
system of care. That identification and services for EI are divided between public health, 
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human services, education, and primary care, and resulted in children not always 
receiving the care they need. 

Grounded in the health and child development research, three trends at the 
national levels created the basis of the early intervention system. Title V of the Social 
Security Act legislates the public health system to support EI, while Part C of the 
Individuals with Disability Act (IDEA) legislates the systems of education and human 
services to provide EI services; all the while, medical associations such as American 
Academy of Pediatrics urge pediatricians and primary care providers to more 
systematically provide screening and referrals for young children who may need EI 
services. The EI system thus spans separate and distinct systems: public health, 
education, human services, and health care (primary care), each with its own unique role 
to play. Figure II. 1 illustrates the three silos at the national, state and local levels that 
exist due to the way the services have been legislated. Given this, interorganizational 
relationships are critical in El's segmented system of services; not only to enable 
organizations within each system to work together efficiently and effectively, but also to 
bridge the separate systems. These interorganizational relationships may provide the 
leverage to improve the EI system for the benefit of children, and act as a model for the 
health care and public health systems as a whole. 
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Figure II. 1 Systems That Make Up the Early Intervention System at the National, 
State and Local Levels. 

Early Intervention and the Public Health System 

National Policy Context : Title V Maternal and Child Health Program of the Social 
Security Act 

Legislated in 1935 as a part of the Social Security Act, the Title V Maternal and 
Child Health Program lays a foundation in public health for ensuring the health of 
women, children and youth (including those with disabilities or special health care needs) 
in America. Among many things, Title V aims to: 

• "Provide and ensure access to preventive and child care services as well as 
rehabilitative services for certain children; 

• Increase the number of children receiving health assessments and follow-up 
diagnostic and treatment services; 

• Implement family-centered, community-based, systems of coordinated care 
for children with special healthcare needs" (MCHB, 2012a, para. 2). 
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Title V established the Maternal and Child Health Bureau (MCHB) at the federal 
level, which aims to "provide leadership, in partnership with key stakeholders, to 
improve the physical and mental health, safety and well-being of the maternal and child 
health (MCH) population which includes all of the nation's women, infants, children, 
adolescents, and their families, including fathers and children with special health care 
needs" (MCHB, 2012b, para. 2). The program plays a critical role in quality oversight, 
coordination (between and among programs and agencies), and overall system capacity 
building (MCHB, 2012b). 
State Policy Implementation 

Maternal and Child Health Services Block Grants (Title V Block grants) are 
awarded to State Title V programs, which are usually located within a maternal and child 
health division of state health departments. Each year, these programs apply for: State 
Formula Block Grants, Special Projects of Regional and National Significance 
(SPRANS), and/or Community Integrated Service Systems (CISS) projects (MCHB, 
2012). 

Such implementation allows states and jurisdictions tremendous discretion in how 
they use of their Title V funds, with the aim to tailor their programs to meet the unique 
needs and challenges in their states. Each state must identify priorities that address the 
needs of their MCH population, and then guide the use of the Maternal and Child Health 
Block Grant funds to those ends. To do so requires a partnership arrangement between 
Federal, State and local entities, and is guided by a state-wide needs assessment 
conducted in collaboration with other organizations in the state and local communities 
every five years. Partnerships are thus a required element of the grant. While discretion 
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exists, each state must use its Title V funds to focus exclusively on the maternal and child 
health population for both infrastructure and service costs. Further, the funding requires 
a match of 75% (meaning that $1 of Federal Title V funding must be matched by at least 
$.75 of state and/or local funding). 

At least 30 percent of federal Title V funds are earmarked for preventive and 
primary care services for children; and at least 30 percent are earmarked for services for 
children with special health care needs, enabling support for early intervention services as 
necessary. The funds may also serve as the 'payer of last resort' for services to children 
without coverage from another program (Catalog of Federal Domestic Assistance 
[CFDA], MCH Block Grants, Number 93.994, 2012). 

A total of 59 states and jurisdictions (including the District of Columbia and U.S. 
Territories) received Title V Maternal and Child Health Block Grant funding in fiscal 
year 201 1, ranging from $145,927 - $41,389,219 per state and jurisdiction, as detailed in 
Table II. 1. The required match results in almost $1 billion being available annually for 
maternal and child health programs at the state and local levels. In fiscal year 201 1, State 
Title V programs served over 39 million individuals. Among the individuals served were 
2.5 million pregnant women, 4.1 million infants, 27.6 million children, and 1.9 million 
children with special health care needs (MCHB, 2012a; CFDA, MCH Block Grants, 
Number 93.994, 2012). 
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Table II. 1 Number of Individuals Served and Annual Appropriations for Title V of 
Social Security Act. 



Federal Fiscal Year 2011 

Number of States and Jurisdictions 59 
Number of Individuals Served 39 million 

Appropriations $546,792 million 

Source: Catalog of Federal Domestic Assistance, 2012. 

Implementation in Colorado 

The Colorado Department of Public Health and Environment's Prevention 
Services receive and distribute the Title V block grants to the 55 local public health 
agencies implementing locally in Colorado's 64 counties. The local public health agency 
staff members are charged with implementing the local action plans in their communities 
(Colorado Department of Public Health and Environment [CDPHE], 2012a). For 
example, the Division of Maternal and Child Health (MCH) at CDPHE is working to 
improve developmental standardized screening and referral rates for all children, 
newborn to age 5, and listed it as a 201 1-2015 priority (CDPHE, 2012a). The Children 
and Youth Branch (CYB) at CDPHE also receives funding from local sources to help 
integrate health into local early childhood systems-building efforts (CDPHE, 2012b). 

Title V was a great start, but as more research citing the significant benefits of 
early intervention was published in the 1970s and 1980s, along with a recognized need 
for improved accessibility to education for individuals with disabilities, an EI system of 
services was legislated in Part C of Individuals with Disabilities Act (IDEA). 
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Early Intervention and the Education and Human Services Systems 

National Policy Context 

The overwhelmingly positive findings in the research for early intervention led to 
legislation mandating EI services in 1986. The Program for Infants and Toddlers with 
Disabilities (Part C of the Individuals with Disabilities Education Act [IDEA]) legislates 
state-based programs of early intervention services for children aged 0-3 who have 
developmental delays and disabilities. This federal law was grounded in a "body of 
existing research [that] clearly underscored the need to intervene early to enable children 
with delays and disabilities to reach their full potential" (Hebbeler, et al 2007, p. 1-1). 

The Education for all Handicapped Children Act of 1975 forged the path towards 
what would eventually become IDEA. In this act, federal funded public schools were 
required to provide equal access to education for children with disabilities. IDEA thus 
focuses on the educational needs of children with disabilities from birth to age 21, which 
is why EI services legislated in the Act are provided through education and social 
services agencies, rather than public health or primary care. The program emphasizes the 
need for cross-agency coordination, again underscoring the need to better understand and 
improve interorganizational relationships. 
State Policy Implementation 

IDEA's Part C provides federal grants to states to implement early intervention 
services such as speech therapy, physiotherapy and occupational therapy for eligible 
children from birth to age three. Again, states have considerable flexibility in use of this 
funding, which helps the states better meet their own unique needs, but also results in 
large variations in the proportion of children enrolled, ranging from one percent to seven 
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percent of early childhood populations in each state (The Early Childhood Technical 

Assistance Center [ECTAC], 2008; Russ, Garro, & Halfon, 2010). In fiscal year 201 1, 

nearly 336,895 children under age 3 were served by IDEA Part C programs, as detailed in 

Table II. 2 (United States Department of Education, 2012). 

Table II.2 Number of Children Served and Annual Appropriations for Part C of 
IDEA. 



Federal Fiscal Year 2012 



Child Count Year (Fall) 20 1 1 

Number of Children Served 336,895 

Percent of Population 2.79% 

Appropriations $442.710 million 

Source: United States Department of Education, 2012. 

Some states have also been able to expand services to children who are at 
risk of disabilities or developmental delays, many of whom come from 
low income families (Russ, Garro, & Halfon, 2010). 
Implementation in Colorado 

The Colorado Department of Human Services, Division for Developmental 
Disabilities administers the Early Intervention Colorado Program and contracts with 
twenty Community Centered Boards (CCBs) statewide to provide these early intervention 
supports and services to children (newborn to age three), and their families (CDHS, 
2012). The CCBs are responsible for determining eligibility, providing case management 
and providing or contracting for services and supports (CHDS, 2012). The CCBs also 
serve youth and adults, outside the focus of the EI system. 
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Limitation in Early Intervention System 

Despite almost three decades of implementation of PART C of IDEA, evidence 
exists that not all children are receiving the care they need. One of the most significant 
barriers lies in the challenge in identifying children who have developmental delays 
during those critical early years of brain development. Kaye, May and Abrams (2006) 
found: 

many young children are not identified with developmental problems until school 
entry or until they demonstrate school failure. . . [and that] although more than 95 
percent of young children see a child health care clinician in the first three years 
of life, most of these clinicians are missing opportunities to detect developmental 
problems, counsel parents of young children about developmental issues, or refer 
children to needed services in the community, (p. 4). 
These findings are supported by the National Survey of Early Childhood Health, which 
found that 94 percent of children had parents who were not getting the guidance or 
education they need as it relates to the screening of their child (Bethell, Reuland, Halfon, 
& Schor, 2004). Dr. Neal Halfon agrees, stating at the Surgeon General's Conference on 
Children's Mental Health in 2000 that "a majority of problems go unrecognized, and most 
children do not receive treatment early in their life unless the problems are severe" (HHS, 
2005 p. 21). Developmental delays are also especially prevalent in young children in low 
income families, and researchers have found that these children are significantly under- 
detected (Kaye et al, 2006). 

In addition to a segmented EI system, America consistently ranks among the 
lowest in cross-country comparisons of child well-being (Wise & Blair, 2007). This is 
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made even more important given the body of evidence that demonstrates the link between 
adverse early life experiences and development, and poorer health or chronic diseases 
later in life (Kuh & Ben-Shlomo, 2004; Shonkoff & Phillips, 2000). Researchers have 
also demonstrated the powerful links between mental health in childhood and later 
economic well-being (Currie, Stabile, Manivong & Roos, 2008). 

Why are some children not receiving the care they need? The systems approach 
introduced in Chapter I illuminates the segmentation in the EI system of care. From this 
perspective, the root of the problem may be that the delivery for early intervention 
services, which is provided through education and human services agencies, is 
disconnected with the primary care system, where pediatricians routinely see children 
birth to three for well-child visits and checkups. Neither education nor human services 
would typically have access to children ages birth-three, unless they are referred to those 
systems (Hebbeler et al, 2007). Identification and referral takes place in other systems, 
specifically the health care system, and to some degree, in early childhood education. 
Early intervention agencies may be well run organizations, but they alone cannot ensure 
that all children who are in need of services receive them: these agencies are set up to 
react to a referral for services. Further, the lead agency that administers the program 
varies among states, as do the funding mechanisms and the collection of programs that 
actually provide the services. This recognition has led to a call to better integrate the 
primary care providers in the health care delivery system into the EI system. 
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Health Care System 

National Context 

The American Academy of Pediatrics (AAP) is a professional membership 
organization of over 60,000 primary care pediatricians, pediatric medical sub-specialists 
and pediatric surgical specialists "dedicated to the health, safety, and well being of 
infants, children, adolescents and young adults" (American Academy of Pediatrics 
[AAP], 2012, Membership, para. 1). The American Academy of Family Physicians 
(AAFP) represents over 100,000 family physicians, family medicine residents, and 
medical students and aims to "transform health care to achieve optimal health for 
everyone" (American Academy of Family Physicians [AAFP], 2012, Vision Statement). 
These two groups of physicians make up a considerable part of the health care system in 
America. 

The AAP (2012, para. 1) aims "to attain optimal physical, mental, and social 
health and well-being for all infants, children, adolescents and young adults", and as 
such, makes recommendations that often become the basis of pediatric health care. The 
organization published a report in 2001 highlighting consensus in the field that pediatric 
clinicians have both the opportunity and expertise to identify children in need of EI 
services to support their development, stating that there "is growing consensus on the 
important role that primary care providers, who see the child on a regular basis and can 
thus assess development over time, can play in recognizing potential developmental 
problems, including social and emotional development problems" (AAP Committee on 
Children with Disabilities, 2001, pp. 192-195). The report goes on to assert that 
pediatricians, with both the opportunity and expertise, are ideal candidates for conducting 
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the ongoing surveillance needed to identify developmental problems (AAP Committee on 
Children with Disabilities, 2001). 

Further, the American Academy of Pediatrics (AAP) also developed the medical 
home concept, which is becoming the standard for provision of high-quality 
comprehensive health care (Long, Bauchner, Sege, Cabral, & Garg, 2012). The medical 
home concept is defined as "model of primary care that is accessible, continuous, 
comprehensive, family-centered, coordinated, compassionate, and culturally effective" 
(Strickland, et al, 201 1, p. 605). The model requires that children have a consistent 
health care provider (a physician or nurse) who works with the family to best meet the 
needs of the child, with access to referrals and care coordination as needed (Strickland, et 
al, 2011). 
Implementation in Colorado 

Many of the professional associations such as AAP and AAFP have active state 
chapters that support individual pediatricians and family medicine physicians practicing 
in Colorado, whether they are based in private partnerships or solo practices throughout 
the state, or work for federally funded VA clinics or HRSA safety net clinics. The 
Colorado Medical Society and other state based organizations also play a key role in the 
primary care system in Colorado, as do physician assistant and nurse practitioner 
organizations. 

Segmented Early Intervention System 

As mentioned above, there are separate steps in the EI process, each taking part in 
a different system. First and foremost is identification and referral, represented as red in 
Figure II. 2, which often takes place within a primary care setting, when initiated by a 
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physician or referred to a physician, or within public health through a safety net clinic or 
home visitation program. There is then the eligibility process that is facilitated by the 
CCBs on behalf of human services. This may also include screening, if not conducted by 
a physician beforehand. If deemed eligible, the CCB provides the EI services, 
represented in purple in the figure below, or contracts therapists to provide the services. 
On the child's fourth birthday, they must transition out of the human services system to 
the education system, represented by the blue arrow below. 



Identification 
and Referral 



Health Care 
System/ Public 
Health System/ 
Early Education 



Elgibility 



Provision 
of EI 

Services 




Education 
Services 



Education 
System 



Figure II.2 Steps in Early Intervention Process, Cross Listed with System. 

Mackrides and Ryherd (201 1, p. 544) found that while 12 - 16 percent of children 
in the United States have a developmental delay, 'as many as one-half of affected 
children will not be identified by the time they enter kindergarten." If developmental 
delays are not detected until they are five, opportunities for developmental gains through 
early intervention are lost. Further, without well defined inter-agency and inter-system 
processes, there are opportunities for a child to be accidentally dropped between the 
systems. 

Finally, a system perspective highlights an extremely important facet of the EI 
system, that no one agency or system can serve the child alone: supporting a child with a 
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disability or developmental delay to reach their full health and developmental potential 
requires successful action in all parts of the systems. 
Identification and Screening: Health care system involvement 

While some families may know that early intervention services will be essential in 
helping their child grow and develop from the time of birth (if the child is born with a 
specific disability that is diagnosed at the hospital, for example), other children may have 
a relatively routine birth, and then develop more slowly or quite differently than other 
children. These children require someone to identify their developmental delay(s) and 
refer them for EI services. 

There is evidence in the literature that "physicians who use an objective screening 
tool will more effectively identify children who may be at risk for, or have, a 
developmental delay than physicians who do not use such a tool" (Rydz, Shevell, 
Majnemer & Oskoui, 2005, p. 4). Several studies also indicate that using a developmental 
screening tool "improves the accuracy with which children are identified when compared 
with decisions based only on clinical judgment" (Rydz et al, 2005, p. 8). However, there 
are indications that pediatricians and family medicine physicians do not regularly use 
standardized tools for reasons that include "a lack of time to administer screens during 
health visits; inadequate compensation; lack of training in the use of specific tools; and 
lack of, or perceived lack of, assessment and treatment resources" (Sices, 2007, p. 9; 
HHS, 2000). Although most physicians report using informal developmental checklists 
as part of their overall strategy of care, a literature review by National Academy of State 
Health Policy found indications that a low percentage of children in need of support for 
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their healthy development are identified, even by physicians (Sices, 2007; Rydz, et al, 
2005). 

The literature on clinical recommendations for screening in primary care is 
inconsistent and often insufficient to direct the primary care physician. In addition, 
multiple barriers exist, including time limited appointments and lack of resources, which 
often prevent physicians from performing initial screening and completing additional 
evaluation and referrals (Mackrides & Ryherd, 201 1; Sices, 2007). 

If a child starts at elementary school with an undiagnosed disability or 
developmental delay, the child stays in the education system to receive special education 
services; however, a critical period of brain development has already passed, and the 
child may not be able to make the therapeutic or clinical advances that might have been 
possible through early intervention (NICHY, 2012). 
Referral to Early Intervention 

When viewing early intervention through a systems approach, it becomes clear 
that improving the screening and referral of children with developmental delays will help 
to overcome some of the segmentation in the system. However, even if screening 
improves through actions of the pediatricians and primary care providers, it will do little 
good for the child if there is no coordination between the health care system and human 
services and education systems. Further to this point, research has shown that providing 
families and clinicians access to resources for assessment and treatment are also critical 
(Kaye, 2006). Some researchers found that "pediatric clinicians [are] reluctant to adopt 
(or continue) using a screening tool unless they [are] confident that the children they 
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identified as potentially needing further care would receive appropriate care (Kaye, 2006, 

p. 26). 

Provision of Early Intervention Services 

As mentioned, the Colorado Department of Human Services, Division for 
Developmental Disabilities administers the Early Intervention Colorado Program in 
Colorado. It provides early intervention supports and services through contracts with 
twenty Community Centered Boards (CCBs) across the state, which in turn, provide the 
services to infants, toddlers, and their families within their communities. The EI system 
in Colorado is visually depicted in Appendix 1. 

It is clear that the EI system in Colorado crosses many systems: public health has 
worked for years to ensure care through maternal and child health programs and 
initiatives, as funding from Title V programs at the national level is provided to CDPHE, 
who in turn, provides it to local health agencies throughout the state. With the creation of 
IDEA, the services for EI are formally legislated and funded, but administered within 
multiple systems: in Colorado, they are housed within human services and education 
systems. Yet, the important time for EI services is birth to age 3, a time before children 
start with the education system. Human services may not have involvement in a child's 
life at all, unless referred. And thus the EI system is also dependent upon the health care 
system, with a significant reliance on primary care providers to screen and identify 
children with developmental delays. Further, there is a new acknowledgement that early 
childhood directors and teachers can also help to indentify children with developmental 
delays. 
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Despite America's relative prosperity in the world, and the substantial health care 
expenditures for seniors (via Medicare), the government makes only a relatively modest 
investment in early childhood development (Bennett, 2008a & Bennett, 2008b). Through 
the health and public health systems that exist, and with coordination with human and 
educational systems, America must be able to leverage funding and resources to be as 
effective as possible for our young children. As discussed, interorganizational 
relationships are a critical element in this endeavor. 
Implementation Challenges 

In summary, systematic reviews of the research on early intervention have shown 
that EI services can enhance physical, emotional, and psychological growth for children. 
Such services have also demonstrated positive short-term and long-term effects for the 
child, family and government systems (Barnett, 1995; Jones & Zigler, 2002). 

The literature also illustrates the highly segmented structure in the delivery of 
early intervention services, which are provided through education and human service 
sectors, while recent consensus emphasizes the parallel role of screening and 
identification as part of primary medical care (King & Glascoe, 2003; Nelson). This 
"heightened awareness of the role of the primary health care system coincides with the 
popularity of the growing research on early brain development, which places child 
development in a biomedical context (Berry, Kutz, Langner & Budetti, 2008, p. 481). 
The recognition that the health care system (including public health's safety net clinics) 
may represent a more logical first point of contact for infants and young children (than 
education and human services) has gained considerable strength, and further emphasizes 
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the need to strengthen and leverage the ties between primary care and public health, and 
between primary care/public health and education and human services systems. 

Recognizing these significant issues, and to stimulate innovative models for 
integrating child development and medical care, the Commonwealth Fund, a private 
foundation dedicated to improving the health care system, sponsors the Assuring Better 
Child Health and Development (ABCD) initiative. The initiative assists states in 
improving the delivery of early child development services by strengthening primary 
health care services and systems that support the healthy development of children from 
newborn to age three (Berry et al, 2008). The ABCD program focuses on building 
system capacity to "increase and enhance the delivery of child development services for 
low-income children through the health care sector" (Berry et al., 2008, p. 32). 
Sponsored by the Commonwealth Fund and administered by the National Academy for 
State Health Policy (NASHP), the program has been operating since 2000 in various 
phases, helping a total of 27 states "create models of service delivery and financing 
through a laboratory for program development and innovation" (National Academy for 
State Health Policy [NASHP], 2012, para. 1). 
Assuring Better Child health Development (ABCD) in Colorado 

ABCD Colorado's vision is that "Colorado's children reach their maximum 
developmental potential" (ABCD, 2012, para. 1). This guides its mission to "encourage 
the use of standardized developmental screening tools in health care settings across 
Colorado to facilitate early identification and referral" (ABCD, 2012, para. 2). 
Colorado's ABCD project started as a pilot project in Arapahoe County in 2005. The 
program then received funding to expand outreach to the entire state two years later, and 
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a state team was developed to guide the efforts. The team includes pediatric primary 
health care providers, parents, and leaders from state organizations and health care 
systems. ABCD Colorado has worked with primary health care providers in over 40 of 
Colorado's 64 counties (ABCD, 2012) since 2005. 

ABCD Colorado aims to support health care physicians by promoting early 
identification of young children with potential developmental delays and referral to 
appropriate community services and resources. It assists primary care practices in 
implementing standardized developmental screening in an efficient and practical way; 
and aims to help a physician practices build and strengthen relationships with early 
intervention resources and services in their community (ABCD, 2012b). It also provides 
training to other professionals who work with children during the critical ages from 
newborn to age three, including child care and early education providers. This might 
include training on developmental milestones, standardized developmental screening 
tools, and appropriate referral processes (ABCD, 2012c). 

ABCD Colorado supported community-driven stakeholder networks to form in 
most of Colorado's 64 counties. Five of these networks are being investigated in this 
study. The stakeholder networks are working to ensure standardized developmental 
screening, referral, and follow up for children in their own local community in Colorado. 
While each group varies in nature due to the community's own unique needs, most of the 
stakeholder networks in this study include the following organizations and agencies, each 
described in greater detail in Chapter IV: 

• Local public health department or agency (PH) 
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• Local Community Centered Board, the EI service provider contracted by the 
Colorado department of Humans services (EI) 

• Local school district or Board of Cooperative Education (ED) 

• Local pediatrician or primary care provider (PC) 

• Local Early Childhood Council (ECC) 

• Local early childhood centers/preschool (EC) 

• Local behavioral health center (BH) 
Conclusion 

In an effort to ensure that all children are able to reach their maximum 
development, the stakeholder networks operating within the EI system provide an 
excellent set of cases to investigate, as the local implementation must be collaborative: 
many organizations must work together with a number of government agencies, private 
health care practices, and community based non-profits to overcome the limitations of a 
segmented system. The ABCD program has a grant funded state team to guide the 
program, and with its leadership, it supports community-driven, autonomous stakeholder 
networks working in most of the state's counties to ensure that every child in Colorado 
has the opportunity to reach their maximum developmental potential (ABCD, 2012). This 
investigation will examine and analyze multiple variables of process, structure and 
connectivity shown to be related to a stakeholder's ability to achieve its goals through a 
comprehensive analysis of five stakeholder networks working to improve the EI system 
in Colorado. The segmentation and limitations of the EI system lends to the importance 
of being able to leverage interorganizational relationships to improve system capacity and 
have a positive over influence of the health of America's young children. 
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CHAPTER III 

LITERATURE REVIEW 

Introduction 

The legislative context and policy implementation of early intervention, as 
described in the preceding chapter, leads to a system where no single organization can 
ensure that children receive necessary early intervention services. An effective and 
efficient system of EI services in Colorado requires primary care providers and early 
childhood educators to identify and refer children who may benefit and be eligible for EI 
services. It also requires that the system incorporate feedback and follow through on 
referrals, a focus of public health. Further, it requires human services and education to 
coordinate an effective transition every time a child turns four and moves from one 
system to the next for continued services. An effective EI system thus require a number 
of organizations and agencies with varied goals, resources, knowledge, and information 
to work together to ensure children's healthy development; highlighting the need for 
effective interoganizational relationships. 
Guided by Theory 

This study examines the questions of 'who works with whom' in stakeholder 
networks working to improve their local EI systems, and how the attributes and 
characteristics of the interorganizational relationships within stakeholder networks impact 
performance. Grounded in a systems approach, this study accepts the fundamental 
assumption of systems theory that a system is greater than the sum of its parts, and 
requires investigation of the whole, rather than specific aspects of a system, issue or 
problem (Von Bertalanffy, 1972). Thus, this investigation focuses on the interrelation of 
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organizations within the system of EI, which exists within the larger context of the health 
and public health systems. More specifically, this approach utilizes the concepts in 
network theories and the applications of network analysis to ultimately examine how 
different attributes and characteristics of interorganizational relationships affect the 
performance of stakeholder networks. 

While the theoretical lens of a systems approach offers great value in its holistic 
perspective of illuminating how and why a system is more than the sum of its parts, 
systems theory requires additional theoretical support in a mixed, method cross case 
research design, such as this one (NCI, 2007). Network theories, which focus on 
relationships between and among actors, are powerful in their mathematical examination 
of relational structure and ties. Network research offers a collection of theories and 
frameworks that correlate the processes through which relationships can exert influence 
on outcomes at the individual, organizational, or network level (Gulati, Nohria & Zaheer 
et al, 2000). Although missing an overarching theory, network research fundamentally 
proves that social structure matters (Ahuja, Soda & Zaheer, 2012). Network research in 
the interorganizational context (hereafter referred to as the network approach), is 
increasingly popular in health and public health research, and provides strength in 
undertaking and explaining not only the interaction, connection and structure of 
organizations partnering in health and public health, but also their affect on performance 
and health outcomes (Wholey, Gregg & Moscovice, 2009; Mabry, 201 1). 

Combined, these frameworks offer a powerful theoretical and methodological 
base for this study. By embedding the network approach within the overarching system 
theory, one can leverage the strengths of each framework, while minimizing their 
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respective weaknesses. To this end, the literature review will reintroduce the research 
questions, follow with a discussion of systems theory and then explore the network 
approach. 
Research Questions 

With an emphasis on interorganizational relationships in health and public health 
systems, and the embedded stakeholder networks' structures and performance, this study 
aims to answer the following research questions: do community based stakeholder 
networks operating within the same context exhibit commonalities in whole- network 
level attributes and characteristics relative to their performance? The second research 
question focuses upon organizational level attributes and characteristics, asking: do the 
relative connectivity and structural positions of member organizations affect stakeholder 
network performance? As described, this study examines stakeholder networks working 
in the context of early intervention, a system that spans health and public health systems, 
as well as human services and education systems. 

Systems Theory 
Overarching Theoretical Framework 

The foundation for systems theory has a long and storied history, traceable back 
to Aristotle's hypothesis that formal nature is of more importance than material nature, 
known in modern times as the simple principle that the whole is greater than the sum of 
its parts (NCI, 2007). General systems theory was developed by Ludwig von Bertalanffy 
(1972), a biologist who crafted the theory to make sense of system characteristics such as 
wholeness, differentiation, and order (Gillies, 1982). These roots of systems theory have 
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informed new theories including chaos theory and nonlinear dynamics, relational 
mathematics, game theory, system dynamics; and network theories. 

Systems theory frames the world as a complex system, composed of subsystems 
that interact with each other. Each subsystem has clearly defined boundaries and 
coherent dynamics (Von Bertalanffy, 1972). The theory can be used to understand the EI 
system, including its structures, processes and outcomes, and their interactions with the 
health care and public health systems. Systems theory is thus used in this investigation as 
an overarching theoretical framework to indentify the elements of the EI system; the 
relationships between these elements; and the system's boundaries and goals. Network 
theories are then used to identify and operationalize the attributes and characteristics of 
the stakeholder networks that include the actors within the system, working to improve 
the system. 

System Characteristics. According to systems theory, most systems have the 
following common characteristics: 

• The elements of a system comprise a whole that is greater than the sum of its 
parts. 

• All systems have elements. These include the goals, as well as the system 
inputs, processes, outputs, feedback, and environment. 

• The structure of systems is defined by its elements and processes. These can 
range from simple to complex. 

• System elements have functional and structural relationships between each 
other and are organized in a way to accomplish a specific goal or set of goals. 
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• To be part of the system, an element must have a relationship with at least one 
element of the system. Any element which has no relationship with any other 
element of the system cannot be a part of that system (Von Bertalanffy, 1972; 
Hayajneh, 2007, p. 3). 

The EI system, viewed through the lens of systems theory, is thus a collection of 
independent but interrelated organizations that are organized in a meaningful way to 
provide early intervention services and supports to children with disabilities or 
developmental delays and their families. They are part of the system's resources (which 
would include therapists with specialist training); its activities (which are primarily the EI 
services for children and supports for their families); and its intended goals (that all 
children in America are able to reach their full developmental potential) (Funnell, 2000; 
Ziviani, Darlington, Feeney, & Head, 2011). 
Application to Early Intervention System in Colorado 

The EI system itself is a component of a larger set of systems, at the nexus of the 
health and public health systems, as well as education and human services. Systems 
theory highlights these systems do not exist in isolation either; they also function in the 
midst of the legislative and judicial systems, the financial and banking systems, and other 
systems that comprise the socio-economic -political system of Colorado (Hayajneh, 
2007); however, that is beyond the focus of this investigation. 

EI is an open system, meaning that it interacts with its environment to fulfill the 
system's overall goal of providing early intervention services and support to children 
with disabilities or developmental delays and their families. Additionally, EI is a complex 
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system, meaning that it includes a number of subsystems such as the EI program's 
network of contracted therapists. 

The EI system is made up of organized relationships that connect different 
elements of the system in specific ways. As detailed in Chapter II, Title Fand Part C 
authorize and require some of these relationships. Others are formal, but not legislated, 
such as the guidance from AAP. 

All systems have a goal or set of goals; that is, they must exist for a purpose. The 
EI system has dual goals of: a) enhancing the development of children with disabilities or 
developmental delays from newborn to age three, while also supporting families' abilities 
to meet their children's needs; and b) reducing costs to the public education and social 
welfare related systems (by reducing the need for special education and minimizing the 
likelihood of institutionalization, poverty, and juvenile delinquency and maximizing 
independent living and developmental potential (Early Childhood Technical Education 
Center [ECTAC], 2008). 

Systems theory identifies four interrelated parts of all systems: inputs, processes, 
outputs and feedback, leading to the system outcomes (Kast & Rosenzweig, 1972). 
These all have attributes that can be measured. Figure III. 1 illustrates the inputs, outputs 
and outcomes in the EI system in Colorado; along with the factors that influence the 
inputs (screening in primary care and/or preschools). Please note that this EI system 
exists across four separate systems (primary care, public health, human services and 
education) as described in Chapter II. 

System inputs are the resources used to produce the outputs. In the EI system in 
Colorado, the inputs include the young children who have developmental delays or 
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disabilities, and their families, along with the therapists with specialized training. All 
systems have processes to convert the inputs into outputs. In EI, services such as 
physical therapy (PT), occupational therapy (OT), and speech therapy are provided. 
Outputs are the services or products that result from the system's processes. An output for 
the EI system in Colorado is that the children receive the support that promotes their 
learning and development. Outcomes are the short and long term impacts, as described 
previously. 

Feedback is information from the system that affects another part of the system. 
For example, if staff or contracted therapists do not have adequate skills or expertise 
working with children with disabilities or developmental delays, there may be ripple 
effects in another part of the system. Further, EI is a dynamic system, so it influences 
and changes its environment, as it is, in turn, being influenced and changed by its 
environment (Hayajneh, 2007). 
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Children with 
developmental 
delays and their 
families. 

If: 

Developmental 
delays are 
recognized and 
children are 
referred for EI 
services from 
primary care 
provider, 
preschool, 
parent, or other. 



=> 



•Staff: PT,OT,SP, 

social work, 

psychology, early 

childhood educators, 

managers, 

administrators. 
•Staff knowledge, 

skills/expertise, time 

and passion, and 

training. 
•Family insight and 

experiences. 
•Policies, procedures 

and guidelines. 
•Funding for Early 

Intervention 

Services. 
•Research on Early 

Intervention. 



^ 



Outputs 



•Children receive 

supports which 

promote their 

learning and 

development. 
•Families receive 

education and support 

regarding their child's 

current skills and 

ways to promote 

further learning and 

development 

(physical, cognitive, 

social and 

emotional). 

•Families receive 
skills in advocating 
for their child, 
supporting the family 
unit and promoting 
empowerment in the 
family. 

•Community 
awareness. 



^ 



Outcomes 



•Children have 

improved skills and 

increased 

participation.. 
•Families feel more 

confident and 

competent in 

supporting their 

children's learning 

and development. 

•Families access their 
chosen community 
services and 
activities. 

•Families experience 
being a part of a well 
functioning EI system 
where children and 
families needs are 
met. 

•Community members 
accept of broader 
range of abilities. 



^ 



Long Term 
Outcomes 



•Children's 
developmental needs 
are met, which is 
reflected in increased 
quality of life, safety, 
health and well being. 

•Families are able to 
advocate for 
themselves. 

•Families have higher 
expectations and 
bigger dreams, and 
can participate in a 
full life. 

•All children are 
accepted for who 
they are. 

•Changes to 
community attitudes, 
legislation, policy 
and funding are 
evident. 



Adapted from Ziviani, et al., 201 1. 



Figure III.l EI System Inputs, Outputs and Outcomes (Colorado). 
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Finally, the EI system in Colorado has functional and structural relationships 
among many of the organizations that are organized in a way to accomplish the goals 
stated above. It is assumed in systems theory that all elements of the system have 
relationships with at least one other element of the system (Von Bertalanffy, 1972). Thus 
a focus on the system cannot be limited by the boundaries of each separate organization; 
rather it is important to investigate whether there are inefficiencies or opportunities for 
improvement in the interorganizational relationships and interdependencies that exist in 
such systems. 
System of Care Literature 

A focus on the system has been emphasized for decades in the system of care 
(SOC) literature, which advocates the need for a coordinated network of services and 
supports across agencies to meet the multiple and complex needs of children with 
disabilities, developmental delays, mental health or other special health care needs 
(Stroul, Blau & Friedman, 2010). Developed to help improve the individualized and 
collaborative services for children with serious emotional disturbances and their families 
(Stroul & Friedman, 1986), the SOC has become a nationally recognized framework for 
effective and family-centered service for individuals and families who need services or 
resources from multiple human service agencies (Snyder, Lawrence & Dodge, 2012). 

The importance of the interorganizational relationships within the system of early 
intervention is highlighted when viewed through the SOC framework, as it requires 
organizations that provide services and support for children with developmental delays to 
communicate, organize, and work together as one coordinated system. In fact, one of the 
guiding principles of a system of care is to "ensure that services are integrated at the 
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system level, with linkages between child-serving agencies and programs across 
administrative and funding boundaries and mechanisms for system-level management, 
coordination, and integrated care management" (Stroul, et al, 2010, p. 6). Thus the 
relationships between organizations within three silos in Figure II. 1 (public health, 
education and human services and primary care) are critical to ensure that long term 
outcomes of the system are achieved. Systems theory and the system of care framework 
illustrate that the EI system cannot simply be viewed as the delivery of services, as that 
view ignores the important interactions between agencies that are so critical for effective 
and family-centered services and supports. The systems view also helps to reinforce that 
all of the organizations within the EI system need to not only deliver quality services, 
they also need to work well together to be able to most effectively help and support 
children with developmental delays to function better at home, in school, in the 
community, and throughout life. It is ultimately about helping children to reach their full 
developmental potential. 

Overall, there is substantial strength in applying a system approach to better 
understand how to improve the health and public health systems. In fact, Mabry and 
colleagues (2008), recently stated that the "health and well-being of the whole population 
may be best conceptualized as a 'systems' problem" (p. 4). This investigation thus 
utilizes systems theory to determine and visualize the parts of the EI system in Colorado. 
The systems lens helps to illuminate the importance of the relationships among the 
different organizations in the context of the system elements: inputs, process, and 
outputs. This approach also helps to inform understanding of the important 
disconnection between the process of identification and referral of children with 
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developmental disabilities and the legislated processes of the EI system (EI services and 
family supports); and identifies the boundary of the system for this study. Further, the 
SOC framework highlights the reasons why such interoganizational relationships are 
important. While a valuable heuristic devise, general systems theory has been criticized 
for its: vagueness, which has made it difficult operationalize without modifications; and 
the assumption that all parts of a system have equal power (Lilienfeld, 1978; Lowe, 
1985). The systems approach provides frameworks for conceptualization in this study, 
but is not adequate to clearly operationalize and measure key variables in this 
investigation, specifically the attributes and characteristics of the stakeholder networks 
working to improve the EI system. Thus network theories and methodologies are used in 
this study to overcome these limitations. 

Given this study's focus on system networks, embedding network theories within 
the overall theoretical framework of systems theory, particularly in the operationalization 
and measurements of the constructs, gives tremendous strength to the theoretical 
conceptualization. There are many network theories and measures developed through the 
network approach that enable this investigation to measure and link the networks' 
attributes and characteristics to performance. Such network theories and literature are 
described in detail in the following sections. 

Network Theories 

Scholars from a variety of disciplines have embraced the conceptual notion of 
networks to study social and interorganizational relationships. The network approach 
builds upon the rational actor assumptions in neo-classical economics, and focuses on the 
relationships among and between actors. Informed by the disciplines of mathematics, 
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management, sociology, psychology, and communication, network scholarship has 
brought value to these fields, as well as to health, political science, public administration, 
management, and public policy, with the development of theory and theoretical 
frameworks, modeling tools, analysis techniques and empirical data. Because of its focus 
on relationships, this line of research has potential to bring tremendous value to health 
and public health systems research, both in theory and practice. 

The research on networks makes up a loosely formed, multidisciplinary body of 
literature with wide-ranging conceptualizations. Despite the variation in wording, 
virtually all definitions of networks contain the common themes of social interaction and 
relationships (Provan, Fish & Sydow, 2007). In this investigation, a network is defined 
as a group of three or more organizations connected in an effort to achieve or facilitate 
achievement of a common goal. The relationships are primarily nonhierarchical among 
actors within a network, and may range from loose connections of information flow to 
formally integrated services to anywhere in between. It is not only the relationships that 
are important in network analyses, but also the absence of relationships, and the 
implications of both for achieving the network goals. 

Unlike research that focuses on organizations, which is framed by organizational 
theory or other traditional guiding framework, there is no one overarching network theory 
to guide organizational network scholarship. Instead, researchers from many disciplines 
have articulated multiple theories to help explain organizational network structure and 
processes. Network research investigates the social structures inherent in health and 
public health systems, defined as "patterns of connectivity within social systems" 
(Wellman, 1988, p. 26). Network scholars posit that organizational networks can be more 
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efficient and effective than agencies working in each of their own silos because they can 
allow for information and resource exchange, and shared efforts, while reducing 
redundancies (Provan & Milward, 1995, 2001; McGuire, 2006; Brass, Galaskiewicz, 
Greve & Tsai, 2004; Alter & Hage, 1993; Agranoff & McGuire, 1999, 2001, 2003; Meier 
& O'Toole, 2001). Further, interoganizational networks have been shown to be a 
successful tool in building community capacity by leveraging the strengths of diverse 
players to solve difficult problems (Agranoff & McGuire, 2003; Provan; Veazie, Staten, 
& Teufel-Shone, 2005). 

In order to understand the concepts underlying the network approach, it is vital to 
have a basic grasp of some of the seminal contributions that build this body of literature, 
and to understand different underlying assumptions and levels of analysis. This review 
is not an attempt to trace the evolution of all network theories through time, nor is it a 
comprehensive review of all network scholarship; rather it focuses on: a) the key facets of 
research on individuals' social networks that informs research on organizational 
networks; and b) organizational network research most relevant to this study and the field 
of health and public health systems. 
Definition of Network 

Network research focuses on the relationships between actors, instead of the 
actors themselves. Brass et al, (2004) define a network as "a set of nodes and the set of 
ties representing some relationship, or lack of relationship, between the nodes" (p. 795). 
A node is the actor within a network, and can be an individual person, organization or 
even a larger group such as a country. 
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In this investigation, organizations are the actors of the stakeholder networks. 
The primary caveat is that organizations consist of individuals, thus the social interaction 
among organizations ultimately occurs primarily between individuals acting on behalf of 
their organizations. Scholars use a variety of terms when referring to the concept of 
networks, including partnerships, collaboratives, collations, but nearly all definitions are 
centered on the common themes of social interaction, relationships, connectedness, 
collaboration, and common goals (Pro van et al, 2007). 

There are different types of networks, with varying goals and scopes, including, 
but not limited to: stakeholder networks, policy networks, service delivery networks, 
project based networks, and innovation networks. This study focuses on goal directed 
stakeholder networks, comprised of organizations in the community with an interest or 
concern with their local system of early intervention. These stakeholders are working 
together to overcome the segmentation that exists in the EI systems to ensure 
standardized developmental screening, referral and follow up for children in their 
communities. 
Assumptions of Network Theories 

Two central assumptions in network theories are important to understand: 
embeddedness and social capital. 

Assumption of Embeddedness. The study of networks first surfaced in the fields of 
anthropology and sociology, as scholars broadened their focus to include the social 
context of the actor. The push behind this new line of research was based upon the 
assumption that "actors do not behave or decide as atoms outside a social context" 
Granovetter (1985, p. 487). Network theories assume that the rational actors within a 

52 



network are connected by a set of ties, and that the structure of the ties affects the actors 
and their relationships (Coleman, 1988; Moreno, 1934; Nadel, 1957; Granovetter ,1973, 
1985, 1992). 

Granovetter (1973, 1985) highlighted the first assumption in network research and 
began the important trend of distinguishing between types of relationships. He argued 
that all economic behavior is embedded in a social context. Granovetter's (1973) 
strength of weak ties theory, as well as Burt's structural holes theory (1992), assume that 
patterns of communication and transactions between organizations may depart from an 
economic assumption of bounded rationality because actors are embedded in a social 
context that influences their decisions and behaviors (Granovetter, 1985). This 
assumption is in line with systems' theory assertions of interdependence and mutual 
influence. 

Assumption of Social Capital. Network theories also have an implicit assumption 
that relationships are a resource in and of themselves. Coleman (1998) explains that 
social capital is 

not a single entity, but a variety of different entities having two characteristics in 
common: They all consist of some aspect of social structures, and they facilitate 
certain actions of individuals who are within the structure. Like other forms of 
capital [physical, financial, and human], social capital is productive, making 
possible the achievement of certain ends that would not be attainable in its 
absence (p. 24). 
Unlike economic or human capital, which are attributes of individual actors, social capital 
is an attribute of the relationship between two actors (Provan & Lemaire, 2012). Thus, 
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when considering actors within a network, their social capital is the potential access to 
information, resources or support inherent in their relationships with the other actors. To 
this end, the concept of social capital provides an important frame of reference in its 
assertion that relationships are productive resources, worthy of investment. This has 
significant application for health and public health systems where a strong return on 
investment may be realized from efforts to strategically strengthen interorganizational 
relationships for improved system performance. 
Theoretical Foundations of the Network Approach 

Much of the early theoretical work in network research focused on the social 
networks of individuals. Moreno (1934) created the structural approach to social 
networks when he developed sociometric models by diagramming relationship networks. 
He aimed to identify patterns of interaction, but ended up sparking a whole new way to 
examine and frame behavior within social contexts. Not only did a whole new line of 
theoretical research emerge, but so too did the current methodological tools of social 
network analysis, which map relationships between actors with use of dynamic-network 
modeling software packages (Scott, 2000; Hill, 2002). Since network analysis focuses on 
relationships between actors, data are generally displayed and analyzed on a matrix of 1 
and 0s that indicate the existence or absence of tie. 

Nadel (1957) expanded upon Moreno's work, arguing the central concept in 
sociological theory should be individuals' roles. To this end, he explored the 
relationships and dynamic between individual actors, their social roles, and the 
environment that structures those roles (Nadel, 1957). This new line of research by 
Moreno and Nadel was different to earlier trends that focused only on the attributes of the 
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individuals, such as physical or human capital, and the aggregation of those attributes. 
Rather, they emphasized the individual relationships and ties among individual actors 
within a network, and gave rise to the focus on relationships, instead of on actors, in 
theoretical exanimations. While the micro focused social network research seeks to 
understand an individual's position in the network for the purpose of maximizing his/her 
own self interest; for example, how an actor's centrality might affect the power of that 
actor; the theories and frameworks have important implications for organizational 
network research. Many of the measurements and correlations, referred to as egocentric 
in the literature, are applicable and consistency hold true in investigations of relationships 
between and among organizations in the literature. With this, scholars crafted a whole 
new way to approach investigations in management, organization theory, political 
science, policy innovation and diffusion, public management and health policy 
implementation. 

Organizational Network Research 
As the public health system includes all organizations providing public health 
services and working toward improved population health, a focus on interorganizational 
relationships is critical in health and public health research. This study is interested in 
structural and relational patterns that may exist across organizational stakeholder 
networks that affect overall network performance. While much of the early network 
research focused on individuals as actors within the network, there is a significant body 
of literature that focuses on organizations as the actors within network (organizational 
networks). 
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Building upon research in intergovernmental relations from political science, and 
based upon the assumptions of social network research described earlier, Gage (1984) 
was the first public management scholar to investigate inter-government networks. He 
sparked a new wave of research investigating the existence and effectiveness of networks 
to deliver intergovernmental programs, and helped to launch one of the newer areas of 
network research: organizational network analysis (Berry et al, 2004; Agranoff, 1986). 

Based upon a review of the literature, many organizational networks share the 
following unique features: 

• They form to achieve a common mission or to deal with complex problems, 
which may be difficult to address without collaboration (Berry et al, 2004). 

• Participation is often voluntary, and is comprised of autonomous actors 
(Weiner, Alexander & Zuckerman, 2000). 

• Network participants often come from diverse organizations, which are 
usually geographically distant (Mitchell & Shortell, 2000). 

• Controlled and regulated by their actors, these networks are usually horizontal 
in shape (Alter & Hage, 1993). 

There has been a huge growth in research focusing on interorganizational 
relations and networks within the field of management in the last couple of decades 
(Alter & Hage, 1993; Sydow, 1998). For example, research has shown that embedded ties 
of ongoing market relationships affect the cost of capital, client relations, and the 
performance of the firms (Uzzi, 1996, 1997, 1999). Other scholars have found that the 
repetitive exchanges protect transactions, while reducing transaction costs (Jarillo, 1993). 
Without a unifying theory, the body of literature on organizational networks is incredibly 
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diverse. Thus, classifying the organizational network research into categories based upon 
level of analysis and focus provides a clarifying framework. 
Typology of Organizational Network Research 

There are three levels of analysis in an organizational network structure: 

1. At the node level of analysis, the actor within a network is the focus. As 
discussed, a network actor can be an individual person, organization or a larger group 
such as a country; however in organizational network research, the actors are always 
organizations. Attributes of organization actors include their sector or field (Borgatti & 
Foster, 2003). 

2. At the dyadic level of analysis, the focus is the relationship between two 
nodes. This is the heart of network research, which focuses on the attributes and 
characteristics of the relationships. Many organizational level variables in network 
research are based upon dyad data because the relationships belong to the organizations. 

3. At the network level of analysis, the focus is on the whole network. 
Measurements often include the overall structure of a network, and can include 
aggregated normative measurements such as trust. This level is referred to as 'whole- 
network' in organizational network literature. 

A four by four matrix to classify the different typologies of network research is 
illustrated in Table III. 1 Typology of Organizational Network Research. Organizational 
network theories actually come from two different, but complementary, perspectives: an 
inward perspective and an outward perspective. Borgatti, Jones & Everett (1998) 
reviewed the literature and found such a distinction, with scholarship separated by 
whether it is focused on "looking within the collective or outside the collectivity" (p. 29). 
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Wasserman and Galaskiewicz (1994) classify this distinction as a micro -level or macro- 
level focus in network research. These distinctions simplify the complex literature, as the 
research also varies by the level of analysis: whole network variables or organizational 
level variables. 
Table III.l Typology of Organizational Network Research. 



Outcome Focus (DV) 



Level of 
Analysis (IV) 



Organizational Outcomes 
(Micro focus) 



Network Outcomes 
(Macro focus) 



Whole Network 
Variables 


Impact of a network on 
member organizations 


Impact of network 
attributes & characteristics 
on network outcomes. 


Organizational 
Variables 


Impact of member 
organizations on other 
member organizations 


Impact of 

organizational/dyadic 
attributes & characteristics 
on network outcomes. 



Adapted from Provan et al., 2007. 

It is important that researchers consider which levels of analysis are most 
appropriate for their investigation, and how those fit within the research framing their 
investigation. This study's first research question fits into the upper right-hand category 
(shaded blue) because it is asking about the impact of whole network variables on 
network performance. This study' second research question fits into the lower right hand 
category (shaded purple) as it is asking about the impact of organizational variables on 
network performance. 
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Impact of Whole-Network Attributes and Characteristics on Network Outcomes 

This study examines stakeholder networks working to improve the system of 
early intervention, first asking: do community based stakeholder networks operating 
within the same context exhibit commonalities in whole- network level attributes and 
characteristics relative to their performance? This question seeks to better understand 
the impact of whole-network characteristics and attributes on performance. 

This is an area of considerable interest in health and public health systems 
research (Consortium, 2012), but is underdeveloped in the network literature (Pro van & 
Lemair, 2012). Many organizational network theories draw upon and use many of the 
ideas and measures developed by researchers focusing on social networks and ego -centric 
measurements; however, the focus is not on the individual actor, but on explaining 
properties and characteristics of the network as a whole (NCI, 2007). For example, 
instead of examining how organizational centrality might affect the performance of an 
actor, this macro-level perspective would focus on how overall network structures and 
processes affect network effectiveness. Characteristics and attributes of the network are 
used to answer questions such as how overall network performance could be improved. 
This perspective assumes that the organizations within a network are working together 
toward a common goal. 

This area of network research is driven by a quest to understand the structures that 
enable networks to be most effective (Provan & Milward, 1991, 1995; O'Toole, 1997; 
Agranoff & McGuire, 1998). Scholars probe a fundamental assumption held by many 
policy officials, hinders, and service professionals that an integrated network of service 
delivery is the most effective approach for providing clients with a continuum of care. 
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This logic can be especially important when serving more vulnerable populations — such 
as individuals with disabilities, the homeless, the elderly, or victims from a natural 
disaster — who may have multiple and significant needs or may need help in navigating 
complicated and fragmented systems of care. 
Network Structure 

Researchers have sought to understand to what extent the structural characteristics 
of a network, such as density or centrality, may enable networks to be most effective 
(Provan & Milward, 1991, 1995; O'Toole, 1997; Agranoff & McGuire, 1998). For 
example, Provan and Milward's (1995) seminal research findings indicate that centrally 
coordinated networks may be more effective than decentralized networks in service 
delivery networks. The authors found that "[njetworks integrated and coordinated 
centrally, through a single core agency, are likely to be more effective than dense, 
cohesive networks integrated in a decentralized way among the organizational providers 
that make up the system" (Provan & Milward, 1995, p. 24). Their model of network 
effectiveness correlates whole network measures of integration with overall network 
effectiveness. Thus, applying Provan and Milward's (1995) model of network 
effectiveness to this study context, one would expect to see a fairly clear pattern in this 
study that higher performing stakeholder networks would have a higher degree of 
centralization and lower performing stakeholder networks would have a lower degree of 
centralization. However, subsequent research that focused on the validation of this work 
has found contradictory findings (Rosenheck et al. 1998). 

Other scholars have sought to understand if there is an ideal level of density for 
network effectiveness. Sandstrom and Carlsson (2008) demonstrated that policy 
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implementation is strongest when network density is high. Further, Agranoff and 
McGuire (1998) found that the density of networks in local economic development 
departments is positively related to achievement of their objectives (adoption of 
economic development policy). However, low density/high centrality networks have 
been found to be an effective structure for innovation networks, but the relationship 
between this whole-network structure and performance has not yet been empirically 
tested (Dhanaraj & Parhke, 2006). Pro van and Milward (2001) also argue that highly 
dense and centralized public service delivery networks work well if the network and 
institutional norms support cooperation and collaboration, highlighting the importance of 
normative network characteristics as well. 

The challenge is that many of these studies varied in their operationalization of 
whole-network: some analyzed density (Sandstrom & Carlsson, 2008; Agranoff & 
McGuire, 1998), while others analyzed a combination of density and centralization 
(Provan & Milward's, 1995; Dhanaraj & Parhke, 2006). Further, the outcome measures 
varied significantly as well, leaving this area of organizational network research still 
fragmented and under developed. Although there is considerable interest, Provan and 
Lemaire (2012) recently stated that there is not yet "research available to inform practice 
about the "right amount" of integration in a network" (p. 643). Given the lack of a 
validated or agreed upon formal theory or model, the study's approach to address the first 
research question is exploratory. 
Network Trust 

While structural measurements are fundamental to network researchers in 
understanding the social structures, research has shown that normative measures, such as 
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trust, are also important network characteristics. Network and collaboration research 
literature emphasize the importance of trust for effective interorganizational relationships 
and networks (Uzzi, 1996; Bryson, Crosby & Stone, 2006; Thomson, Perry & Miller, 
2008a). To this end, trust is a "common theme in virtually all definitions of 
interorganizational networks" (Provan et al, 2007, p. 643), with empirical evidence that 
overall trust is an integral condition for network performance (Uzzi, 1996; Provan et al, 
2003; Huxham & Vangen, 2005). 

Trust is considered a central and ongoing requirement for successful collaboration 
because it enables quality work, while reducing transaction costs (Huxham & Vangen, 
2005). Putnam (1995) emphasizes the importance of trust to "facilitate coordination and 
cooperation for mutual benefit" (p 67), while Bryson, Crosby and Stone (2006) assert that 
trusting relationships "are both the lubricant and the glue- they facilitate the work and 
they hold collaboration together" (p. 48). Trust is also related to network sustainability, 
as trusting interorganizational relationships can substitute for legal contracts and formal 
organizational agreements on a limited basis, such as between contract periods (Ring & 
Van de Ven, 1994; Thomson, Perry & Miller, 2008b). Further, findings in the network 
evaluation literature also emphasize that the effectiveness of networks can be influenced 
greatly by intra-group dynamics, including trust (Hill, 2002). For example, a network 
with low trust may implode before the purpose is achieved. Alternatively, a long 
standing network group may have greater trust levels and therefore may be more 
effective with limited resources (Hill, 2002). 

Although there is little disagreement on the necessity of trust for effective 
collaboration, the literature offers many ways to conceptualize it. One recent study in 
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public health networks found that a trusted partner was described as: a) being reliable and 
following through; b) sharing the network's goal or mission; and c) being willing to 
engage in an open and honest discussion (Varda et al, 2008). 

Building upon findings in the literature that overall trust at the network level is 
positively correlated with network performance (Uzzi, 1996; Provan et al, 2003), and is 
necessary for networks to perform at high levels (Zaheer, McEvily, & Perrone, 1998; 
Huxham & Vangen, 2005; Ring & Van de Ven, 1994; Bryson, Crosby & Stone, 2006; 
Thomson et al, 2008b), one would expect to see a fairly clear pattern of higher trust in 
the higher performing stakeholder networks, and a lower trust in the lower performing 
stakeholder networks. 
Impact of Organizational Attributes and Characteristics on Network Outcomes 

This study also investigates different organizational level attributes and 
characteristics, including the connectivity and roles of the organizations; asking in the 
second research question: do the relative connectivity and structural positions of member 
organizations affect stakeholder network performance? In order to answer this research 
question, each organization's dyadic relationships must be analyzed in each stakeholder 
network, and then compared across networks to confirm if an association exists between 
dyadic relationships and overall network performance. 
Organization Positions within Networks 

Early network research emphasized the importance of an actor's position in a 
network. Freeman (1979) examined the role of an individual's position within a network, 
and the affect on his or her power. This perspective is also found in structuralist position 
theory (Wellman, 1988). Further, Ahuja (2000) found that an actor's degree centrality 
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(the number of ties an actor has in a network) increases his information available to him, 
while George, Zahra, Wheatley, and Khan (2001) found a positive relationship between 
actor degree centrality and the actor's capabilities and level of learning. 

Burt (1997) also provides an important frame of reference in his assertion that 
relationships are not only productive resources, but that well structured networks can be 
more effective than large inclusive networks. Further, O 'Toole and Meier (2001) posit a 
relationship between network structure and service-delivery performance: they found in 
their examination of educational programs that "performance improves in districts where 
superintendents engage in more network interactions, even if one controls for a variety of 
factors that affect this performance" (p. 291). 

Applying this early network research to organizational networks, an organization 
may be in a position of power due to its structural position between two other actors, but 
on the other hand, it may have the added responsibility of maintaining the ties between 
organizations that are not directly connected in order for the network to function 
effectively (NCI, 2007). Identifying organizations that hold a position between other 
organizations is one way to identify influential actors in networks (Borgatti et al, 2002). 
This may have significant implications for the ability to leverage certain organizations or 
structural positions to improve network performance. It would also be important to 
consider whether these organizations were spanning structural holes, and thus acting as 
brokers in the network as well. 
Organization Roles within Networks 

Regardless of the overall degree of centralization (or decentralization) of the 
network, there are often one or more actors that have greater influence in a network. Burt 
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(1995, 1997, 2000) found that the more areas of the network that an actor has ties with, 
the greater the potential information or resources they have access to. Findings in the 
literature also indicate that clusters of organizations or fragmentation in a network affect 
the flow of communication, resources or knowledge, and may create redundancies or 
efficiencies, depending upon the structure and purpose of the network (Burt, 1995). For 
example, a fragmented network, that includes unconnected organizations, dyads, and 
subgroups, has many structural holes, which has implications for its actors and its 
effectiveness. Burt's (1995) theory of structural holes posits that there are two 
advantages to an actor who bridges the structural holes: a) the actor can control the 
information and/or broker the relationships; b) the actor has access to new or non- 
redundant information. This theory has been regularly applied to organizational 
networks, even though his original research focused on social networks of individuals. 
Further, brokerage roles are frequently correlated with influence in an organizational 
network (Kilduff & Tsai, 2003). 

Kilduff and Tsai (2003) also found that the role of broker in networks is advisable 
only for actors "who have legitimacy in the social context.... Actors who are considered 
outsides, or who are from non-traditional groups, may be punished for attempting to span 
across structural holes" (p. 58). Given the research setting of EI, one would expect that 
local health agencies would carry this legitimacy with the members of the stakeholder 
networks, enabling the network to use the efficiencies of subgroups, without the loss of 
information flow, resources and diffusion. Early childhood education providers and 
school districts may also be appropriate brokers, but because they come from education, 
rather than health systems, they may be considered outsiders. 
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Connectors within Networks 

Krackhardt (1994) also found that the degree of connectedness, which he defined 
as "the extent to which actors are able to connect to each other through the network," is 
an important aspect in network research (in Kilduff & Tsai, 2003, p. 38). Building upon 
the importance of connectivity in organizational network research, Varda et al.'s (2008) 
research into the core connectivity of public health organizations led to a method to 
measure the connectivity of organizations that incorporates the quality of their 
relationships, as well as the quantity. This is indicated by: a) an organization's centrality; 
b) the level of trust that other organizations have for them; and c) how much they value 
their relationships with the other organizations (Varda, 2012). 

Adding a new consideration to network research, Varda et al. (2008) found that 
the most valuable member of the collaborative is considered by public health leaders to 
be "one that has a credible, well connected presence in the community, can devote 
resources (time and money), and contributes the effort to make things happen." (p. E4). 
From their findings, the researchers developed three dimensions for assessing the value 
of network members: a) power and influence; b) active involvement; and c) contribution 
of resources. 
Strength of Ties within Networks 

Granovetter (1973) posited in his strength of weak ties theory that actors have 
'strong ties' within networks (close relationships) as well as 'weak ties' (acquaintance 
relationships); and it is the weak ties that can become a greater resource to the actor. For 
example, it is through the weak ties that information is diffused and disseminated and 
exchanges are more varied. If one was confined to just a network of strong ties, that 
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actor would be excluded from exchanges in the world around him (Granovetter, 1973). 
That said, strong ties also have benefits because they can be trusted sources of influence. 

Also differentiating between the types of relationships, Krackhardt (2010) found 
that strong ties (as opposed to weak ties) influenced the outcome of a union election 
because of the trust inherent in the strong ties; while Hansen (1999) found that strong ties 
are more useful to effectively transfer information. Further, Uzzi (1997) found that strong 
ties provide greater problem solving capacity. Thus strong ties signal trusted sources of 
advice and may be more influential in uncertain or conflicting situation. 

However, network research demonstrates that it is unfeasible and unsustainable 
for organizations to maintain numerous relationships (Kilduff & Tsai, 2003 ; Monge & 
Contractor, 2003). This would especially be the case for physicians, as their primary 
focus is providing individual care. (It is both the focus of their training and the method 
through which they are reimbursed for services). This has a significant application in 
public health practice, where state and local public health system performance is 
assessed, in part, by counting the number of stakeholders in partnerships (NPHPSP, 
2007). Through interviews with public health leaders, Varda et al, (2008) found a similar 
consensus that "it is difficult to maintain interactions between all members of a 
collaborative, and that if this was achievable, it probably was not sustainable." (p.E4). 
Therefore, they posit that "subgroups within a collaborative are efficient. . . and frequent 
interaction is not always necessarily good for connectivity" (Varda et al, 2008, p. E4). 
Taken together, it is critical that primary care providers are engaged in the network to be 
able to bridge the segmentation that exists in EI, but simultaneously unfeasible to expect 
that they could build and sustain many relationships. 
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Fortunately, network research tells us that being well connected does not mean 
that an organization needs to have ties with all members of a network. As with any 
research of interorganizational relationships, it is important to consider the context of the 
relationships under investigation. In this study, the organizations banded together to 
ensure standardized developmental screening, referral and follow up for children in their 
communities. These networks were mobilized to bridge a gap between public health, 
primary care and education to overcome the segmented system of identification and 
delivery for early intervention (EI) services. It is vital that primary care physicians are 
well connected and engaged in the networks in this study, thus one would expect to see 
primary care providers have strong ties with early intervention and public health in the 
high performing networks, and weak ties with other organizations in the networks. 
Network Analysis in Health and Public Health Systems Research 

Network research in public health is generally focused on organizational 
networks, with the organizations that make up the networks a blend of government 
agencies, other public institutions, private health care providers and non-profit, 
community-based organizations. Network research in health and public health is 
distinctive from traditional approaches in the field in a number of important ways: first 
and foremost, network research focuses on relations and patterns of relations, rather than 
on specific attributes of the organizations operating within the health and public health 
systems. Second, since network research can be carried out at various levels of analysis, 
network analysis can be used to investigate associations between whole network 
variables of a health focused network system improvement and/or health outcomes; and 
between organizational level variables and system improvement and/or health outcomes, 
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as in this study. Third, network research can integrate both quantitative and qualitative 
data, as in this investigation, and thus enable greater richness in the analysis for better 
understanding of the affect on health systems or population health outcomes (Kilduff & 
Tsai, 2003). 

The network approach lends itself very well to health and public health services 
and systems research (PHSSR). Given that the public health system includes all 
government, private and nonprofit organizations contributing to the core functions of 
public health, as described in Chapter II, the network approach offers a "promising 
approach for capturing the interorganizational relations that form public health systems" 
(Who ley et al, 2009, p. 184). 

Merrill and colleagues' (2008) research on local health departments' networks 
was extended by Wholey et al. (2009), who found that while local health departments 
were relatively central, the health departments were usually not the most central 
organization in community-based organizational networks. The researchers emphasize 
the importance of the network's context, such as community size. Along the same vein, 
Mays and colleagues (2006) found that the size of jurisdiction served by a local health 
department was a strong predictor of performance, with larger jurisdictions performing 
better than their smaller counterparts. 

Very little of the network research in health and public health systems research 
examines the relationship between network attributes and characteristics and performance 
or impact on health outcomes. Suen, Christenson, Cooper and Taylor (1995) showed that 
centralized local health departments had better performance measures than those with 
decentralized or hybrid structure, but stayed focused on the structure of the organizations, 
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rather than their networks. Hyde and Shortell (2012) found in their systematic review of 
public health systems research that the relationships between organizational structure and 
health outcomes is complex, calling research that focuses on interorganizational 
relationships one of the most notable gaps in the literature. Part of the complexity is that 
there is no single organization that can carry out the ten essential public health services 
by itself. Archival and survey based studies of public health systems may have an 
implicit focus on the local health department as the lead and most central organization, 
which may not accurately reflect the system (Wholey et al, 2009). These types of studies 
may also be unable to capture the intangible characteristics and attributes of the 
relationships within the complex systems. 

Depending upon the type of data that is collected, it is possible to examine a range 
of questions that shed light on the interorganizational relationships that exist in health and 
public health systems, including: the overall level of commitment or influence among 
organizations in the network; patterns of involvement; the connection of specific types of 
organizations to others; the quality or types interactions and connections; and the 
directionality and strength of the relationships; along with the level of trust of 
organizations within the network, or the network as a whole. For example, network 
research has been used in studies of mental health networks to explore such structural 
issues and then compare the findings with those of other networks providing similar 
services (NCI, 2007). 

The systems level perspective offered by network theories is also important for 
health and public health research (Mabry, 201 1; Mabry, Olster, Morgan, & Abrams, 
2008). Networks are inherent in public health, where public, private and nonprofit 
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organizations work together to assure the conditions for population health. There is a 
void of evidence regarding the effectiveness of networks. The grey literature on 
partnerships and collaboration in health and public health includes descriptive case 
studies of many different initiatives and interventions aimed at improving population 
health in some way. However, this collection of descriptive works highlights a 
fundamental challenge within this area of research: most partnerships and collaboratives 
are locally driven, formed to address a specific barrier, improve access or implement a 
specific intervention in their local community. Few studies of networks in health and 
public health are generalizable. Significant reasons for this are the unique variations 
within the public health systems between, and among, counties and states. Consequently, 
this study aims to build upon established systems and network research to investigate 
patterns of relationships within the health and public health systems, specifically within 
stakeholder networks working to overcome the segmented EI system, with the aim to 
create new generalizable knowledge. 
Network Performance 

Although the two bodies of network literature, social networks and organizational 
networks, inform this study's operationalization of network performance, there is 
additional complexity that must be taken into account when measuring performance in 
health and public health. As discussed in Chapter II, early intervention (EI) efforts to 
offer physical, occupational, and speech therapies to young children with developmental 
delays or disabilities have been shown to drastically improve fine and gross motor 
development, as well as readiness for school, which then can have positive impacts on the 
child's entire life (Jones & Ziglar, 2002; Haskins, 1989). However, for a stakeholder 
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network working within the EI system, how can one best measure performance? A count 
of the number of children receiving EI could be one measure, but that would hardly 
capture the complexity of the situation. Who is to say that the network was directly 
responsible for each child receiving early intervention? Further, being referred to EI may 
not guarantee that the child will benefit if other parts of the system do not function 
properly. Given the networks' goals, how can one measure the impact of EI to the lives 
of children? If the stakeholder network were responsible for changing just one child's 
life, is that not of tremendous value to that family? Does reducing the number of barriers 
faced by the families working within the segmented system count as success, even if the 
system is still left partially segmented? Would creating awareness about the importance 
of standardized screening count as success, even if the increased awareness was not 
directly tied to children being referred? All of these questions highlight the tremendous 
difficulty in which a statistical count can in no way begin to identify collaboration 
success or failure. It is unlikely a single approach to measure the performance of such a 
network would be agreed upon in the literature. In fact, some scholars argue that 
collaborations should not be identified as successful or not successful unless key 
indicators point in that direction over time and in varying contexts (Bingham, 2003). 

Review of the network literature shows that scholars are focusing upon the three 
structural levels of analysis discussed above- node, dyad and network, but when focusing 
on evaluation of performance, some scholars also measure network performance by 
looking to community outcomes. For example, Pro van and Milward (1995) measured 
changes in the incidence of their problem in question and aggregated indicators of client 
wellbeing. However, Roussos and Fawcett (2000) lament difficulties with this 
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measurement because "visible changes in population-level outcomes take longer than the 
lifetime of many [networks]" (p. 374). 

Consideration of performance literature outside the realm of networks clarifies the 
definition of performance measurement. Performance measurement is defined in 
business journals as the process "of quantifying the efficiency and effectiveness of 
action" (Neely, Gregory, & Platts, 1995, p. 1229). Thus a performance measure for the 
stakeholder networks is a metric or set of metrics that quantify the efficiency and/or 
effectiveness of the network's actions (Neely, et al, 1995). The General Accounting 
Office (GAO, 2012) defines performance measurement as: 

the ongoing monitoring and reporting of program accomplishments, particularly 
progress towards pre-established goals. ... Performance measures may address the 
type or level of program activities conducted (process), the direct products and 
services delivered by a program (outputs), and/or the results of those products and 
services (outcomes). A program may be any activity, project, function, or policy 
that has an identifiable purpose or set of objectives, (p 3). 

Thus applying the two definitions, performance in this study focuses on the most 
fundamental question: has the stakeholder network achieved the objectives it was formed 
to achieve, addressing both process and outcome measures. 

Collaboration researchers often measure the effectiveness as perceived by its 
members (Thomson et al, 2008a); while some network scholars have also asked the 
network customer and/or client to rate the network's goals achievement (Pro van & 
Milward, 1995). Responding to the challenges and warning in the literature on 
measuring performance, as described in the next chapter on Methodology. 
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Conclusion 

Many of the many network attributes and characteristics addressed throughout 
this literature review are not apparent to the organizations within a network because they 
evolve over time, influenced not only from the decisions of that actor, but also from the 
decisions and perceptions of all the other actors within the network. The structures 
illuminated by network analysis "constitute social realities of which the social actors 
themselves may not be aware. . . . [thus] network research has an emancipatory potential in 
that it can inform actors of non-obvious constraints and opportunities inherent in patterns 
of social connections" (Kilduff & Tsai, 2003, p. 23). Thus answering the research 
questions posed in this study requires a structured and robust methodology that draws 
upon network analysis instruments, tools and techniques, as detailed in the following 
chapter. 
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CHAPTER IV 
METHODOLOGY 
Research Design 
Overview 

This chapter provides a detailed description of the design and procedures used in 
the study. The methods have a sound epistemological grounding in system and network 
theories, and the procedures are consistent with the network approach. Aiming to 
understand how patterns of interaction, and other attributes and characteristics of 
community-based stakeholder networks affect their performance, this study utilized a 
mixed method, most-different cross-case research design. 

The research design has two phases. Based on the methodology of the network 
approach, Phase 1 focused on the characteristics and attributes of five stakeholder 
networks. Whole network and organizational/dyadic variables were examined through 
social network analysis (SNA), and analyzed with performance measures based on 
stakeholder interviews, self assessments, and local early intervention assessments. 
Within case analyses was conducted for each network, and then the findings were further 
investigated through cross case analysis to identify whether patterns between independent 
variables and performance were observed. 

Phase 2 used content analysis to investigate the patterns that emerged from the 
cross case analysis, to better understand how network attributes and characteristics affect 
network performance. Network findings informed the coding process and the 
construction of categories. Specifically, content analysis was used to better understand 
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an emergent distinction in the relationship between key network actors and network 
performance. 

Each network was treated as a bounded case study, thus allowing for systematic 
comparison (Yin, 2003). George and Bennett (2005) argue that "there is growing 
consensus that the strongest means of drawing inferences from case studies is the use of a 
combination of within-case analysis and cross-case comparison within a single case study 
or research program" (p. 18). As the purpose of the study is to identify patterns in the 
attributes and characteristics of networks, relative to their place on a performance 
continuum, the cross-case method with a most-different design is a strong choice. 

The most-different cross-case research design deliberately seeks to compare cases 
that are contrasting or different from each other in order to find similar processes or 
outcomes across cases (Przeworski & Teune, 1970). The most different design approach 
can indicate the robustness of a relationship that is observed between the independent 
variables and the dependent variable. Thus, the approach lends strength to the findings 
since such an observed relationship would be consistent across contrasting cases (Faure, 
1994). This case-oriented approach thus emphasizes diversity in the selection of cases 
(George & Bennett, 2005). 

Focusing on the dynamic interactions of public health agencies with public and 
private organizations that affect health, the five stakeholder groups are goal-directed 
networks, working to ensure standardized developmental screening, referral and follow 
up for children in their Colorado communities. The stakeholder networks were mobilized 
to overcome the segmented system of early intervention services in their local 
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community. The networks were purposefully selected through key informant meetings to 
emphasize diversity in the selection of cases. 

Social network analysis can be used to measure variables through confirmed and 
unconfirmed ties. Interorganizational relationships are not a one way endeavor, and 
analytical methods, such as network analysis capture that complexity. 

Identifying the Data Needed 

This study investigated both whole network variables and organizational level 
variables, as the independent variables, in the context of network performance, the 
dependant variable. This study first focused on the impact of whole network variables, 
including density, centralization and trust, on network performance, in an exploratory 
approach. This study's second question focused on the impact of organizational level 
variables, including relative connectivity, brokerage and strength of ties, on network 
performance. 
Network Performance Data 

An understanding of network performance was required for investigation into 
both research questions. The stakeholder networks aimed to ensure standardized 
developmental screening, referral and follow up for children in their community, with the 
ultimate aim to enable "Colorado's children [to] reach their maximum developmental 
potential" (ABCD 2012, para. 2). Although it is not possible to measure the network's 
exact contribution toward this goal and long term outcome, data on three separate 
performance assessments were collected to triangulate performance for each of the 
stakeholder networks: an external assessment by ABCD Colorado leadership team; 
network members' self assessment; and an early intervention assessment. The three 
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separate measurements were used to enhance construct validity and reliability of the 
performance measure (Yin, 1994). The rationale and identification of data needed to 
answer this study's research questions follows, with the variables detailed in the 
Variables and Measurements section later in this chapter. 

External Assessment. It is important to understand how the networks were 
performing from the perspective of an actor outside of the network, but with an intimate 
knowledge of the local EI systems and necessary efforts towards systems improvement. 
The Assuring Better Child health Development (ABCD) leadership team fit that bill, and 
developed the 'ABCD Model Community' Guide: a set of deliverables to help inform 
stakeholders' efforts and the measure their progress. The ABCD Model Community 
guide is a process measure of performance, informed by ABCD Colorado's experience 
about standardized developmental screening, and the referral and follow-through process 
that should occur once children are identified. The guide also takes the factors that need 
to be considered when multiple community organizations are performing and advocating 
for standardized screenings into account (ABCD, 201 1). Given the difficulties in 
accomplishing and measuring systems building related efforts, this external assessment 
provides an important perspective of network performance. The six key activities and 
supporting action steps identified in the guidelines are detailed in the Variables and 
Measurements section of this chapter. 

Network Self Assessment. Network member's own perception of success is 
advocated in the literature as another important perceptive of performance. Asking each 
member of each stakeholder network to rate how successful the network's focus been for 
children in their community captures this important perspective. 
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Early Intervention Assessment. It is also important to capture an objective 
assessment in the triangulated performance measure. There is no available data on 
screening rates at the community level; however, an indicator that children with 
developmental delays or disabilities are being identified early, and the referral and follow 
up processes between agencies are working to ensure that children are receiving early 
intervention services, was needed. The researcher looked to Early Intervention Colorado 
for such an indicator. The degree to which each local community meets the EI targets 
where the stakeholder networks operates was calculated from the local EI reporting data, 
and was the third measure used to triangulate network performance. 

After identifying the performance data needed, this investigator next focused on 
the characteristics and attributes of the networks. This required relational data for each of 
the stakeholder networks, meaning that all organizations in each networks needed to 
provide information about their relationships with all of the other organizations within the 
network. 
Whole-Network Data 

The study's first research question sought to better understand the impact of 
whole-network characteristics and attributes on performance, asking: do community 
based stakeholder networks operating within the same context exhibit commonalities in 
whole- network level attributes and characteristics relative to their performance? This 
is an area of considerable interest in health and public health systems research, but is 
underdeveloped in the network literature. Informed by the literature, an exploratory 
approach was selected. Network density was identified as one of the whole network 
independent variables, with the aim to investigate whether there was a relationship 
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between the overall density of ties in stakeholder networks and network performance. 
Further, it was important to understand the extent to which organizations were more or 
less connected in the overall network structure. Degree centralization was selected as a 
measure to capture the way the network was structured, and how centralized or 
decentralized it was, with the aim to investigate whether there was a relationship between 
network centralization and network performance (Pro van & Lemaire, 2012). These whole 
network measures and respective equations are detailed later in the chapter. 
Organizational and Dyadic Data 

The study's second research question inquired whether any relationships existed 
between attributes and characteristics at the organizational level and network 
performance. Thus data at the organizational level were identified, collected and 
analyzed. Based upon the literature, the organizational level data that capture network 
actors' structural positions include: degree centrality, relative connectivity, closeness 
centrality, betweenness centrality and brokerage relationships (Provan & Lemaire, 2012). 
These variables are described in detail in the Variables and Measurements section of this 
chapter. Network literature also indicates that a 'well connected organization' is not the 
same as the 'most connected organization'. Thus, the type of relationships between 
primary care providers and public health and early intervention organizations is 
important. For this area of study, it was important that the relational data not simply 
indicate an awareness of the organizations or frequency of contact; rather, it needed to 
capture the intentional efforts of organizations to enhance the capacity of the primary 
care providers to implement and carry out standardized screening and strengthen the 
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follow through process. To this end, the measure of coordinated ties was selected as an 
organizational variable and is described in the measurement section. 

The Phase 2 content analysis further explored emergent findings from the network 
analyses. This mixed method, cross case research design is summarized in Table IV. 1 ; 
with the participants, variables and analysis detailed in the following sections. 
Table IV. 1 Summary of Research Design. 
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Data were collected via online surveys from organizations from all five 
stakeholder networks detailed in the following sections. 
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Participants and Sites 

Focusing on the dynamic interactions of public health agencies with public and 
private organizations that affect public health, the population of this study includes five 
stakeholder networks working to ensure standardized developmental screening, referral 
and follow up for children in their communities. As detailed, these networks were 
mobilized to overcome the segmented system of identification and delivery for early 
intervention services in Colorado. 
Selection of Stakeholder Networks 

The five stakeholder networks included in the study were strategically selected 
based upon key informant meetings with the Colorado Assuring Better Child health 
Development (ABCD) lead team. This investigator employed purposeful sampling of 
stakeholder networks guided by theory and specific criteria. Creswell (2007, p. 118) 
explains purposeful sampling as the selection of an "intentional sample. . .that can best 
inform the researcher about the research problem under examination." In this study, each 
stakeholder network was selected from specific criteria to ensure it is consistent with 
network theories, and to ensure a broad spectrum of performance in which to be able to 
compare and contrast across cases. 

The first five criteria were based upon Bogason and Zolner's (2007) five 
characteristics of networks: 

1 . a relatively stable horizontal articulation of interdependent, but operationally 
autonomous actors; 

2. who interact through negotiations; 
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3. which take place within a common regulative, normative, cognitive 
framework; 

4. that is self regulating within limits set by external agencies; and 

5. which contributes to the production of public purpose" (p. 6). 
Additionally, the following criteria for the selection of stakeholder networks were added 
for this study to ensure similarities among networks, except on the dependent variable. 
Each network needed to: 

6. have a core vision to improve child developmental heath, either directly or 
through its social or economic determinants,; 

7. have formed as part of the ABCD initiative because no one organization felt 
that they could address the objectives on their own; 

8. have been in existence for at least one year; and 

9. be working to ensure standardized developmental screening, referral and 
follow up for children in their Colorado community. 

Comparing networks is a difficult task, as they often vary tremendously in their 
characteristics and attributes. Because they form to overcome specific challenges or 
problem or meet the needs of a specific community or system, they often vary in purpose, 
scope, membership, and size. Therefore, it was important that these networks developed 
for the same purpose, in the same state, and with relatively levels of resource 
munificence. 

The first nine criteria narrowed down the number of possible stakeholder 
networks to 35. The five stakeholder networks in the study were then selected from these 
35 based upon an external assessment by the Colorado ABCD lead team in a most- 
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different research design (Przeworski & Teune, 1970); selecting two stakeholder 
networks from states with the highest assessments from ABCD, two stakeholder 
networks with lower assessments from ABCD and one stakeholder network in between. 
This final criterion helped to ensure that the five networks exemplified a range of 
performance. This informant team has been leading the ABCD initiative in Colorado 
since 2007, providing technical assistance and support to community based stakeholder 
groups working to increase standardized developmental screening and improve the 
system of referral and follow up for children in Colorado. The ABCD team knows the 
networks and communities very well, but was able to provide an objective assessment 
because it is not a part of any one network. As such, the team developed an assessment 
matrix, based upon the six key activities determined by ABCD to be essential in 
successfully working to ensure standardized developmental screening, referral and follow 
up for children. 

A total of five stakeholder networks were selected, since five is the maximum 
number of cases recommended in a single qualitative comparative case study (Cresswell, 
2007). The key activities are summarized in Table IV. 2, with the full assessment criteria 
detailed in Appendix 2. 

Table IV.2 Assessed Stakeholder Activities. 

Network activities central to goal achievement 

1. Developing community partnerships; 

2. Creating a strong foundation; 

3. Carrying out well defined and organized physician outreach; 

4. Clearly defining the referral and follow up process; 

5. Making parent education a priority; and 

6. Setting up trainings. 
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The stakeholder networks in this study operate in five counties in Colorado. The 
specific stakeholder network names and communities are confidential in this study, 
although the counties range in population from 14,548 people to 636,963 people, and 
5,330 households to 227,151 households (US Census, 2011 estimate). As diverse as their 
population size, the land area in square miles also ranges considerably, from 798 square 
miles to 2,386.10 square miles. Their population densities range from 8 people per 
square mile to 717 people per square mile, while the median household annual incomes 
(2006-2010) range from $40,699 to $60,433 per household (US Census, 2010). 
Stakeholder Network Members 

To address the needs of their communities, each network includes a mix of 
government, private and nonprofit organizations operating within the health care system, 
public health system, education system, and system of human services, including the 
early intervention system in their community. Each of the five networks includes the 
following types of organizations: 

Primary Care Providers: Anywhere from one to eight primary care pediatricians 
or family medicine physicians are part of the stakeholder networks working to ensure 
standardized developmental screening, referral and follow up for children in their 
community in Colorado. These primary care physicians have not traditionally been a part 
of the EI system in Colorado, given that EI services are administered by the CDHS, and 
provided by CCB therapists or contactors; however, there is strong growing consensus 
about the importance of pediatric and primary care physicians doing standardized 
screening with tools such as the Ages and Stages Questionnaire (ASQ) to identify 
children with developmental delays, or at risk of developmental delays. The published 
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mission statements of the providers vary, but most detail a primary goal of providing the 
highest quality and most comprehensive pediatric health care available in a private- 
practice setting. Central to the identification phase of EI, these primary care physicians 
bring the perspective of the health care system to the stakeholder network; however, their 
primary focus is on providing quality individual care to their patients. 

Local Public Health Agencies (PHs): The Colorado Department of Public Health 
and Environment's vision for 'Healthy People, Healthy Families, Thriving Communities' 
guides the 55 local health agencies or departments in each of Colorado's 64 counties 
(CDPHE, 2012). PH agencies provide a broad spectrum of services to address public 
health and safety issues. They offer services and programs mandated by state statute, 
federal/state funding, as well as those that reflect the unique needs of their community. 
As such, PH agencies are delivering services and programs to meet 201 1-2015 Colorado 
MCH Priorities identified in the Title V federal funding. Further, The Child, Adolescent, 
and School Health (CASH) unit at CDPHE also works with local public health agencies 
to develop and implement strategies to improve the health of children and adolescents in 
their communities (CDPHE, 2012a). 

The local PH agency within each community is a part of the stakeholder network. 
Guided by CDPHE's emphasis on comprehensive, coordinated strategies, the agency's 
primary focus in early intervention is on capacity building, and strengthening 
partnerships and collaboration with organizations and resources that exist in the 
community to reach its EI related goals. 

Community Centered Boards: The Colorado Department of Human Services 
(CDHS), Division for Developmental Disabilities (DDD) is the state office that "provides 
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leadership for the direction, funding, and operation of services to persons with 
developmental disabilities within Colorado" (CDHS, 2012, para 4). CDHS implements 
Part C of IDEA for the State of Colorado by authorizing Community Centered Boards 
(CCBs) to be responsible for the provision of early intervention services, along with other 
support and services for individuals with developmental disabilities. CCBs, in partnership 
with private service providers and therapists, manage and deliver services to individuals 
with developmental disabilities and their families throughout the state. The twenty CCBs 
serve approximately 1 1,000 (adults and children) and their families in every county 
across the state (CDHS, 2012). 

As the EI service provider, the local CCB is also a part of the stakeholder network 
in each community. Focused on providing services, the CCB's primary focus is on the 
provision of quality care and services to individuals with disabilities or developmental 
delays. 

Early Childhood Councils: According to authorizing legislation, the role of 
Colorado's 30 Early Childhood Councils is to "improve and sustain the availability, 
accessibility, capacity and quality of early childhood services for children and families 
throughout the state" (CDE, 2012, para. 1). Early childhood services are defined by the 
state legislation as including: early learning; family support and parent education; social, 
emotional, and mental health; and health (CDE, 2012). 

Early Childhood Councils across Colorado bring together partners from each of 
these service areas "to coordinate, increase and improve the education, health, 
social/emotional, and parenting opportunities available to all young children and their 
families" (CDE, 2012, para. 3). 
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Educational Organizations that are a part of the stakeholder network can include 
School Districts, which oversee the local public education system within their district; 
Local Head Start Organizations, which provide early learning opportunities for children 
from families with low incomes; and Boards of Cooperative Educational Services 
(BOCES), which are an extension of the local school districts. BOCES provide services 
to children, families and school staff where economies of scale are achieved by delivering 
across the district rather than through individual schools (BOCES, 2012). 

Child Care Providers and Early Learning Centers/Preschools are also part of the 
stakeholder network in each community. 

The behavioral health or community mental health center in the community is 
part of each stakeholder network as well. It aims to offer "comprehensive and accessible 
mental health and substance abuse treatment, housing, education and employment 
services for adults and the leading resource of treatment for children, teens and families" 
(MHCD, 2012, para. 3). These organizations thus bring a mental health perspective. 

Other organizations that may be included in the stakeholder networks include 
health care therapists or home visitation programs, faith based organizations and 
nonprofits operating the local communities. 
Census of Ties 

The population in this study is comprised of all organizations that are a part of the 
five stakeholder networks. This approach is consistent with network theories as it takes 
"a census of ties in a population of actors, rather than a sample" (Hanneman & Riddle, 
2005, ch. 1), meaning that the data will contain the entire population of relationships 
bounded within each collaborative. 
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Human Subjects Review 

Prior to the start of recruitment and data collection, this investigation was 
reviewed and approved by the Colorado Multiple Institutional Review Board. As per the 
Colorado Multiple Institutional Review Board (COMIRB) policies and University of 
Colorado (CU) investigator responsibilities, the investigator completed the Collaborative 
Institutional Training Initiative (CITI) Human Subjects Protection Training Course, and 
submitted the research proposal to COMIRB for review and acceptance before 
recruitment of data collection. Although the research subjects involved in the project are 
living, this investigator determined that the project could be appropriately reviewed as 
non-human subject research or be granted exemption under Category B, since the 
research involved the use of survey procedures focused on organizations (COMIRB, 
2012). The application described the research questions, purpose and methodology, 
recruitment, populations, and analysis; declared funding from the National Coordinating 
Center for Public Health Services and Systems Research and Robert Wood Johnson 
Foundation; and declared that no conflict of interest existed. The full application is 
attached in Appendix 3. The investigation was approved by COMIRB as non-human 
subject research. 
Recruitment of Subject Population 

Upon approval by COMIRB and completion of the network selection, the 
investigator presented the details and implications of the study to the ABCD Colorado 
lead. Information about the study was also provided to the liaisons for each community 
in which the five stakeholder networks operated. The ABCD liaisons, in turn, emailed 
the stakeholder members to convey this information and let them know to expect an 
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invitation from the investigator. This was an important series of steps to create 
awareness, leverage ABCD's unique position, and enable entre to the stakeholder 
networks. 
Data Collection 

The network survey was the primary data collection vehicle in this study. The 
survey questions were based upon Varda's (2012b) PARTNER Tool, a validated 
instrument for network analysis. The comprehensive research that went into the 
development of the tool, and its extensive use in the field, minimized the need for pilot 
testing. Adaptations and additions to the survey were made to tailor the language to the 
research setting. The consent, recruitment material and survey questions were presented 
to the ABCD lead team for information and feedback, and modifications were made as 
necessary. 

The investigator worked with ABCD staff and community liaisons to get contact 
information for a representative from each organization in each of the five networks. An 
email invitation to participate in the network survey was then sent out to all identified 
organizational representatives in the networks. The email contained further information 
about the study; the consent form; investigator contact details; and a unique login and 
password to the online PARTNER survey, shown in Figure IV. 1 . The specific language 
in the invitation is attached in Appendix 4. The investigator followed up via email in 
three to four days, and again as necessary. 
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Figure IV. 1 Screenshot of PARTNERTool Survey Login. 

Respondents were asked to complete the surveys as a representative of their 
organization. When respondents logged in, consent was required before the questions 
were displayed. The first set of survey questions then requested information about the 
organization and its role in the network, as well as the existence (or non-existence) of 
relationships with all other organizations in the network. Building upon that list, the 
network questions asked each respondent about the types and frequency of interaction 
between their organizations and all the other organizations in their list. These types of 
questions produce data that can be analyzed to determine strength or weakness of ties, 
connectedness, and measures of centrality, as well as determining brokerage roles within 
the network. Other questions asked about the level of commitment of organizations to 
the network's mission, or the reliability of organizations in their work, to determine trust 
and value at the node, dyad and network levels. Questions also asked respondents for 
their organization's perception of network outcomes and success, capturing the second of 
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the three dimensions of network performance in this study. The full survey instrument is 
attached in Appendix 5. 

Privacy, Confidentiality and Minimal Risk. Information about the organizations 
involved in the stakeholder networks in each of the five communities was collected 
through the PARTNER Tool at: http://www.partnertool.net/. Data were maintained in the 
PARTNER Tool program for the duration of the study. Survey data were also exported 
from the PARTNER Tool, and maintained on an encrypted laptop, in password protected 
files. 

This investigation met the regulatory definition of minimal risk because the 
probability and magnitude of harm or discomfort anticipated in the research were not 
greater in and of themselves than those ordinarily encountered in daily working life 
(COMIRB, 2012). The only research activities that involved human subjects were 
through the use of survey procedures. There was no physical or psychological 
examination, test or intervention. The survey was completely voluntary, and since it was 
completed online, it could be started and finished at anytime that best suited their 
schedule. Further, all findings are presented as an aggregate score for an organization, or 
at the network level. 

Variables and Measures 
Network Performance- Dependent Variable 

Investigation into both research questions requires an understanding of network 
performance, the dependent variable in this investigation. Three separate measures were 
collected and analyzed to triangulate performance scores for each of the stakeholder 
networks: external stakeholder assessments, self assessment, and local early intervention 

92 



assessment. All three measures were summed with equal weight to determine each 
network's performance index. 
External Assessment 

The Assuring Better Child health Development (ABCD) leadership team 
developed a step-by-step guide of tangible deliverables to help inform community efforts 
and the measure their progress with this work. The six key activities and supporting 
action steps towards achieving a model ABCD community are summarized below: 

1 . Developing community partnerships. Deliverables: partners identified and roles 
defined and agreed upon. 

2. Creating a strong foundation. Deliverables: comparison of actual system with the 
ideal system in their community; collaboration; safety net strategies; measurement 
and evaluation; and funding. 

3. Carrying out well defined physician outreach. Deliverables: review and 
verification of local health care provider data; prioritization of health care 
provider outreach; development of a plan to initiate and/or strengthen and sustain 
outreach efforts; health care provider messaging; and use data to support 
physician outreach and technical assistance. 

4. Clearly defining referral and follow through process. Deliverables: review of state 
referral algorithm; agreement on the local process; partners understand their roles 
and relationships within the algorithm; partners understand where to refer children 
who "flag"; partners understand where to refer child who do not flag but still may 
need extra help; partners are aware of and understand Early Intervention Colorado 
referral process and referral form; partners are aware of and understand the Child 
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Find referral process and referral form; partners have clear expectations from 
referral process and understand referral status update form; partners use data to 
inform and support community referral efforts. 

5. Making parent education a priority. Deliverables: parent education group active 
and engaged; three priorities for parent education identifies and pursued. 

6. Setting up trainings. Deliverables: training delivered on developmental screening 
tools, child development, and referral processes as appropriate (ABCD, 201 1). 

The ABCD Colorado leadership team assessed the performance of the stakeholder 

networks according to the guidelines and deliverables described above. Each group was 

given an overall scored based upon points earned for each of the six steps and 

deliverables. 

Network Self Assessment 

Members of the stakeholder networks were also asked to self assess the network's 
efforts in the survey, asked: from your organization 's point of view, how successful has 
this focus been for children in your community? Respondents were asked to answer on a 
rating scale ranging from 'Not Successful; Somewhat Successful; Successful; Very 
Successful; Completely Successful'. The answers were weighted from 1= Not Successful 
to 5= Completely Successful and averaged for a self assessment score. 
Early Intervention Colorado Assessment 

Part C of IDEA requires that the lead agency in each state monitor and supervise 
all early intervention services and programs within the state (EI Colorado, 201 1). 
Although it is difficult to tie child outcomes (such as the improved acquisition and use of 
knowledge of skills) to network performance, it is important to incorporate an element of 
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the system performance, since the network aims to improve the system. Thus, a measure 
of the system output is the third measure used to triangulate network performance. 

The Colorado Department of Human Services is the lead agency for EI in 
Colorado, and is required to report performance data from local programs in relation to 
the state targets as identified in the Colorado State Performance Plan: Federal IDEA, Part 
C Services. Indicator 6 in the Colorado State Performance Plan: Federal IDEA, Part C 
Services for 2010 assesses the percentage of children from birth to age 3 with 
Individualized Family Service Plans (IFSP) in the EI system, as compared with the 
statewide target (EI Colorado, 2012). The data are available on the Early Intervention 
Colorado website by early intervention program under: Monitoring Reports, Plans of 
Correction, Public Performance and Determinations. The degree to which each local 
community meets its targets where the stakeholder networks operate was calculated as 
the third dimension for network performance. 
Network Demographics 

Stakeholder network demographics were also measured, including: size, measured 
by the number of organizations in each network; age: measured by the number of years 
the organizations have been working together in a focused effort to ensure standardized 
developmental screening, referral and follow up for children in their community; and 
relative county size, measured by the population estimates from the 2010 census. 
Whole-Network Independent Variables 
Density 

Density is a core measurement of network structure, which measures the number 
of total ties in the network. The density of a network is the total number of ties divided 
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by the total number of possible ties (Borgatti et al, 2002). This measurement helps to 
illuminate the overall connectedness among organizations in the network, from a strictly 
numbers point of view. All organizations in each stakeholder network where asked: from 
the list, please select the organizations/programs with which you have an established 
relationship (either formal or informal) (Survey Question 10). Relationships were 
determined based upon the respondents' selections of the other organizations. A whole- 
network measure of density D was calculated for each stakeholder network, where / is the 
actual number of ties, and is divided by the total possible ties n(n — l)/2 (where n is the 
number of actors in the network). 

Density (Scott, 2000): 

I 
D 



n(n — l)/2 

This measurement helps to illuminate the overall connectedness among 
organizations in the network, from a strictly numbers point of view. This measurement 
allows for an easy understanding of the connectedness of a network; however, it should 
be used cautiously when comparing networks of different sizes given that the 
measurement is sensitive to the number of actors (Scott, 2000). 
Degree Centralization 

Degree centralization expresses how tightly the network is organized around its 
most central point (Freeman, 1979). This measure complements network density by 
measuring how the network is organized, and whether it has a centralized structure. 
Higher centralization scores indicate that the network is more centralized, meaning that 
the network is organized around one or a few highly central organizations. Lower 
centralization scores indicate that the network is less centralized, meaning that the 
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network may have ties spread more evenly among actors (Scott, 2000). UCINET 

computes the network degree centralization of a network, where Max is the maximum 

possible value and C Di is the degree centrality of node Di and n is the number of actors in 

the network. 

Degree Centralization (Freeman, 1979): 

CD= Y.{Max{C Di )-C Di ) 
n 2 — 3n + 2 

Network Trust 

As detailed in literature review, organizational network and collaboration 
literature emphasize the importance of trust for effective interorganizational relationships 
and networks (Uzzi, 1996; Bryson, Crosby & Stone, 2006; Thomson at el, 2008b). To 
this end, trust was analyzed at the whole network level and compared across cases to 
investigate whether a positive relationship between trust and network performance 
existed. Requiring a measure of whole network trust, this study employs Varda et al's 
(2008) three dimensions of trust (reliability, share a common vision and openness to 
discussion). Whole network measurement of trust was measured by the average trust 
score of all three dimensions of trust in the network. Specifically, organizations in each 
stakeholder network were asked: 

How reliable is the organization/program (in the context of standardized 
developmental screening, referral and follow up for children in your community)? 
(Survey Question 16); 

To what extent does the organization/program share a common vision of 

standardized developmental screening, referral and follow up for children in your 

community? (Survey Question 17); and 
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How open to discussion is the organization/program (in the context of 
standardized developmental screening, referral and follow up for children in your 
community)? (Survey Question 18). 

Answers were requested on a scale of 1 to 4 (l=Not at all; 2= a small amount; 3= 
a fair amount; 4= a great deal). The sum of the three responses from each organization in 
each of the stakeholder networks were then divided by the highest possible trust between 
all organizations (12 points). 
Organizational Independent Variables 

The study's second research question is relevant to an area of organizational 
network research that focuses on the impact of an organization's relational and structural 
attributes and characteristics, measured at the organization/dyad level, as opposed to the 
whole network level. The variables are informed by an established area of network 
theory that asserts correlations between organizational level variables and overall 
network outcomes. 
Degree Centrality 

Degree centrality is the extent into which an actor in a network is directly 
connected to the other actors in the network (Freeman, 1979). Central organizations have 
more connections to others in the network. The degree centrality C' D (p k ) is count of the 
degree for a given point (p k ) on the network graph. C D (p k ) = £f =1 a(p i( p fe ) where 
a(jp t , p k ) =1 if pj and p k are connected and if they are not connected. This is divided 
by the maximum degree centrality, where p k can be adjacent to n — 1 other points in the 
graph (Freeman, 1979). 
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Degree Centrality (Freeman, 1979): 

C d (Pk) = : 

n — 1 

Based upon the relationships respondents designated in the network survey, the 
measure was calculated for all organizations in each of the five stakeholder networks. 
Betweenness Centrality 

Betweenness centrality reflects the extent to which an organization is on the 
'shortest path' between two other organizations, and can thus moderate and influence the 
information between them. Where gtjQpk) is the number of geodesies g t j linking two 
points, Pi and p k that contain p^, the betweenness centrality of an actor C' B (p k ) is a sum 
of it partial betweenness values for all unordered pairs of actors where i#j#k (Freeman, 
1979). Betweenness is therefore a measure of the number of times a point occurs on a 
geodesic. The normalized betweenness centrality is the betweenness divided by the 
maximum possible betweenness expressed as a percentage (Borgatti et al., 2002). 
Betweenness Centrality (Freeman, 1979): 

gijQPk) 



c' B (jPk) = Y 



i>j 9ij 

Betweenness centrality is used to investigate whether an organization takes on the 
role of a gatekeeper within the network, calculated by the extent that an organization is a 
tie between other organizations. For example, if the same type of organization has high 
measures of betweenness centrality in the high performing networks in this study, this 
may provide an important clue worth further investigation and consideration. 
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Brokerage 

Brokerage occurs when, in a group of three or more organizations, one 
organization requires another organization to reach a third organization. (More 
specifically, given A, B and C, brokerage occurs when A has a tie to B, and B has a tie to 
C, but A has no tie to C. A needs B to reach C, and B is therefore a broker) (Borgatti et 
al, 2002). UCINET calculates the brokerage measure proposed by Gould and Fernandez 
(1989) by counting the number of times that each organization is a broker in all group 
possibilities within the network. Brokerage was calculated for each organization based 
upon a response of monthly or more to the question: At this point in time, how frequently 
does your organization/program work with this organization on issues related to 
standardized developmental screening, referral and follow up for children in your 
community? (Survey Question 11). This was calculated for each organization within 
each stakeholder network. 

Each of these variables were measured at the organizational/dyadic level and 
compared across cases to investigate whether patterns between organizational level 
variables and network performance existed. 
Relative Connectivity 

Building upon centrality measures, relative connectivity focuses on both the 
number and quality of connections between organizations in each network. The measure 
is a function of: a) each organization's degree centrality; b) the level of trust that other 
organizations within the network have for them; and c) how much the organization values 
the relationships with the other organizations. These three measurements are described 
below: 
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a) Degree Centrality. Degree centrality is measured by the number of direct 
ties that an organization has within a network. Key to this study, this measurement also 
allows investigation into whether an organization is in a central position, or is connected 
along the periphery of a network. 

b) Trust. The whole-network variable of trust described in the previous 
section is an aggregate of the trust measures for each organization in each network. 
Thus, the three dimensions of trust measured at the organizational level can be analyzed 
at the organizational level and whole-network level. For the trust measure, organizations 
were asked to rate all of the other organizations' levels of reliability, mission congruence, 
and openness to discussion on a scale of one to four: 

• reliability: this organization is reliable in terms of following through on 
commitments; 

• mission congruence: this organization shares a common vision of the end 
goal of what working together should accomplish; 

• openness to discussion: this organization is willing to engage in frank, 
open and civil discussion (especially when disagreement exists) (Varda, 
2012a). 

These results were summed and a mean trust score was calculated for each organization 
in each stakeholder network. 

c) Value. These dimensions of value are measured in this study and make up 
the third element of relative connectivity. Organizations were also asked to rate each 
organization's power/influence, level of involvement and, resource contribution on a 
scale of one to four: 
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• power/influence: the organization holds a prominent position in the 
community be being powerful, having influence, success as a change 
agent, and showing leadership; 

• level of involvement: the organization is strongly committed and active in 
the partnership and gets things done. 

• contributing resources: the organization brings resources to the 
partnership like funding, information, or other resources (Varda, 2012a). 

These results were summed and a mean value score was calculated for each organization 
in each stakeholder network. Relative connectivity was the sum of an organization's 
degree centrally; mean trust score; and mean value score, relative to the other 
organizations in the network. One organization in each network is attributed the highest 
relative connectivity of 100 percent. 
Relationship Strength 

Connectivity of the primary care physicians was conceptualized by the type of 
activities for which they maintain relationships with other organizations in the network. 
Members of each stakeholder network were asked not only about their relationships and 
frequency of contact with the other organizations in the network, but also about the 
activities that the relationships entailed. All organizations were asked: what kinds of 
activities does your organization/program 's relationship with this organization/program 
entail (in the context of standardized developmental screening, referral and follow up for 
children in your community)? (Survey Question 12). Respondents answered on a rating 
scale: 

• None (coded 0); 

102 



• Cooperative Activities: This may include exchanging information, closing the 
loop on referrals, attending meetings together, and offering resources to partners 
(coded 1); 

• Coordinated Activities: Includes cooperative activities in addition to intentional 
efforts to enhance each other's capacity for the mutual benefit of programs (coded 
2); and 

• Integrated Activities: In addition to cooperative and coordinated activities, this is 
the act of using commonalities to create a unified center of knowledge and 
programming that supports work in related content areas (coded 3). 

The primary care provider (PC) relationships that entailed coordinated activities were 
integral. To this end, PCs' strong ties, as indicated by coordinated activities, were 
measured and compared across cases to investigate whether a relationship between 
these ties and network performance existed. 

The conceptual and operational definitions of all major variables are detailed in 
Table IV.3. 
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Table IV.3 Conceptual and Operational Definitions -Variables and Measures. 



Demograpphic Variables Operationalized Measure 


Size of the network 


Number of organizations actively involved in the stakeholder network. 


Age of network 


Length of time the stakeholder network has existed. 


County size 


Total population of county where the stakeholder network operates. 


,. . ,, _ ,. . . Theoretical link Operational Definition Operationalized Measure 
Variables Definition 


Performance 


External 
assessment 


Perceived and actual 

achievement of goals is 

a measure of network 

outcome (Gray, 1989, 

2000; Provan & 

Milward, 1995) 


ABCD's assessment of each 
stakeholder network's 


Network's overall score on model community guide, as 
compared to other networks. 


Members' self 
assessment 


Survey responses recorded on 
scales ranging from 1 (not at all) 
to 5 (to a great extent)/ coding. 


Overall, how effective has the stakeholder network been at 
reaching its objectives? 

Please indicate which of the following outcomes you feel have 
been achieved as a result of the efforts (selection of stakeholder 
network's objectives- as many that apply). 

From your perspective, which of the above describes the most 
successful outcome of the stakeholder network (selection of one 
of the collaborative objectives)? 

What aspects of collaboration contribute to this success? 


EI assessment 


Percentage of children with 
IFSPs, the written plan for EI 
services from the local EI 
Colorado program, as compared 
to state target. 


Early Intervention Colorado local services data for children, 
newborn through three years of age with IFSP, as compared to 
state target. 
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Table IV.3 (cont.) 



, . ... _ ~ £. Theoretical link Operational Definition Operationalized Measure 
Variables Definition r r 


Whole Network Variables 








Network 
structure 


Density 


Centralization related to 
network effectiveness 
(Provan & Milward, 
1995). 


Network structure calculated via 
SNA through UCINET 
program- index created from 
two network measurements 
collected through survey. 


The number of ties in a network as a percentage, in relation to 
the total number of possible ties. 


Degree 
centralization 


How similar or dissimilar the organizations are in terms of their 
number of connections to others. 




Trust 


High levels of trust 
correlated with network 
and collaboration 
effectiveness (Huxham 
& Vangen, 2005). 


Aggregated mean score from 
organizational level trust data 


See in Organizational Variables. 


Organizational Variables 








Connectivity 


Degree centrality 


Strategic connections 
can improve the network 
functionality and 
outcomes (Burt, 1995; 
Freeman, 1979; 
Granovetter, 1973; 
Krackhardt, 2010; 
Kilduff&Tsai, 2003; 
Varda et al., 2008). 


Network interaction calculated 
via SNA through UCINET 6 
program and PARTNER Tool. 


The number of connections that organizations have to other 
members of the networks. 


Betweenness 
centrality 


The extent to which an organization is on the 'shortest path' 
between two other organizations. 


Brokerage 


The extent to which an organization connects otherwise 
unconnected organizations in a network. 


Relative 
connectivity 


The highest number of quality of connections. 


Strength of ties 


The number of ties representing coordinated activities: 
intentional efforts to enhance each other's capacity for the 
mutual benefit of programs. 


Value 


Power/influence 


The levels of 
organizational values 
can affect effectiveness; 
not just power as 
traditionally thought, but 
also involvement and 
time contribution (Varda 
et al., 2008). 


Organizational value score 
calculated from survey 
responses recorded on scales 
ranging from 1 (not at all) to 4 
(to a great extent). 


The organization holds a prominent position in the community 
be being powerful, having influence, success as a change agent, 
and showing leadership. 


Level of 
involvement 


The organization is strongly committed and active in the 
stakeholder network and gets things done. 


Resource 
contribution 


The organization brings resources to the stakeholder network 
like funding, information, or other resources. 
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Table IV.3 (cont.) 



Trust 


Reliability 


High levels of trust 
correlated with network 
and collaboration 
effectiveness (Huxham 
& Vangen, 2005); Three 
dimensions of trust: 
'reliability', 'in support 
of project' and 'open to 
discussion' (Varda et al., 
2008). 


Trust score calculated from 
survey responses recorded on 
scales ranging from 1 (not at all) 
to 4 (to a great deal). 


This organization is reliable in terms of following through on 
commitments. 


In support of 
project 


This organization shares a common vision of the end goal of 
what working together should accomplish. 


Open to discussion 


This organization is willing to engage in frank, open and civil 
discussion (especially when disagreement exists). 
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Data Analysis 

Analysis of data collected in Phase 1 is based upon the methodology of network 
analysis. As detailed in the previous chapter, network analysis enables the stakeholder 
networks to be measured and analyzed mathematically, producing calculations of the 
overall network structure and the attributes and characteristics of the relationships 
embedded within, as well as producing a visual map (graph) (Scott, 2000; Wasserman & 
Faust, 1994). Network analysis provided a powerful and precise method to visually depict 
a network; and an excellent analytical tool to calculate and map the network attributes 
and characteristics (Scott, 2000). 
Network Analysis 

Given the systems approach required to answer the research question, network 
analysis was undertaken to graph and calculate key attributes and characteristics of the 
five stakeholder networks. Graphs are useful to present information visually, enabling 
one to see patterns that may not otherwise be evident. Network matrices also enable the 
use of complex mathematical algorithms to calculate patterns in attributes and 
characteristics (Hanneman & Riddle, 2005). 

In order to answer the research questions, binary measures of relations had to be 
calculated for each network separately from the survey data. At the most basic level, 
respondents from each stakeholder network were asked: from the list, please select the 
organizations/programs with which you have an established relationship (either formal 
or informal) (Survey Question 10). The responses were assigned numbers to distinguish 
between the presence or absence of a tie (coded one or zero, respectively). Multiple- 
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category nominal measures of relationships were also collected and analyzed. For 
example, respondents were asked: at this point in time, how frequently does your 
organization/program work with this organization/program on issues related to 
standardized developmental screening, referral and follow up for children in your 
community? (Survey Question 1 1). Respondents were asked to scale their relations from 
the list below to gather information about the strength of their ties: 

• Never/We only interact on issues unrelated to standardized developmental 
screening, referral and follow up; 

• Once a year or less; 

• Every few months; 

• Every month; 

• Every few weeks; or 

• Every week. 

Further, other questions asked about the kinds of activities that the relationships 
entailed. For these, each relationship was coded on a qualitative scale by its type, rather 
than its strength. Unlike the true-false nature of the binary nominal data, the multiple 
category nominal measure is multiple choice (Hanneman & Riddle, 2005). Normative 
information, such as trust and value, was also collected, given the importance of these 
characteristics in the literature. 

Since network analysis is based on a census, rather than a population sample, it is 
always important to attain a response rate as close to 100% as possible. While no finite 
limits of survey response rate have been determined in the network research to date, 
esteemed network scholar Stephen Borgatti (2012) asserted that a response rate of close 
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to 70% is important to maintain research validity and reliability in network research. The 
stakeholder network response rates in this study were as follows: 

• Alpha: 10 of 12 organizations responded for an 83% response rate; 

• Bravo: 19 or 27 organizations responded for a 70% response rate; 

• Charlie: 8 of 10 organizations responded for an 80% response rate; 

• Delta: 12 of 18 organizations responded for a 67% response rate; and 

• Echo: 10 of 14 organizations responded for a 71% response rate. 
The PARTNER Tool (Varda, 2009b), UCINET 6 (Borgatti et al, 2002) and 

NetDraw (Borgatti et al, 2002) are the software programs used to analyze the survey 
data, and to create the graphical representations of the networks. The survey responses 
were stored in the PARTNER Tool, since respondents completed the survey online 
directly through the PARTNER'S web based platform. Functions within the tool 
displayed the raw survey data in Excel files. Preliminary data analysis was conducted 
using PARTER Tool functions to convert the relational data into network visualizations, 
and to complete calculations of network measurements including relative connectivity, 
trust and value. The binary and value data for each stakeholder network were then 
manually converted from the PARTNER Tool into edgelist text files and imported into 
UCINET for further analysis. UCINET converted these files into sociomatrices. Entries 
code a relationship from the organizations in the row to the organization in the column. 
By way of example, a matrix of relational data for one of the stakeholder networks is 
depicted in Figure IV.2. 
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Figure IV.2 Sample Matrix of Stakeholder Network. 

[Valued for frequency of relationship]. 

This matrix depicts the presence or absence of relationships, based upon 
responses by each organization to survey question 1 1 (coded: Never/We only interact on 
issues unrelated to standardized developmental screening, referral and follow up= 0; 
Once a year or less= 1; Every few months= 2; Every month = 3; Every few weeks = 4; 
Every week= 5; Every Day= 6. Sociomatrices were created to represent network ties for 
each of the relational, value and trust questions in each of the stakeholder networks. 

UCINET was then used to calculate the whole network measurements of density 
and centralization, as well as organizational measurements including centrality, 
betweenness centrality, and brokerage. NetDraw was used to develop additional network 
graphs and visualizations. 

Finally, a cross case analysis was conducted to look across cases, and led to new 

generalizations from emergent patterns between the network attributes and characteristics 

and network performance. The stakeholder networks were placed along a continuum of 

performance based upon their aggregate performance measures. The investigator then 
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examined each of the independent variables relative to network's place along the 
performance continuum. 

The cross case study design is ideal for "uncovering the meaning of a 
phenomenon for those involved" (Merriam, 2009, p. 5). Further, case study analysis is 
well suited for research questions focused on "discovery rather than confirmation" 
(Merriam, 2009, p. 19). Further, one of the greatest strengths of case study methodology 
is precisely in this challenge: in "achieving high conceptual clarity... where statistical 
methods and formal models are weak" (George & Bennett, 2005, p. 19). Finally, case 
study methodology allows for multiple cases to be compared, revealing insight into the 
similarities and differences among cases, enabling generalizations to develop through 
analysis (Creswell, 2007). 
Content Analysis 

A follow up phase to the network analysis employed a qualitative methodology to 
help capture the rich detail of the attributes and characteristics being explored. 
Qualitative methodology is a strong fit for this phase of this study, as "qualitative 
procedures provide a means of accessing unquantifiable facts... how humans arrange 
themselves and their settings and how inhabitants of these settings make sense of their 
surroundings" (Berg, 2004, p. 7). 

NVivo 9 software was used for the coding process and qualitative analysis. As per 
qualitative analysis practice, these categories were mutually exclusive and "conceptually 
congruent" (Merriam, 2009, p. 186). Cresswell (2007, p. 148) articulates that "data 
analysis in qualitative research consists of preparing and organizing the data ...for 
analysis, then reducing the data into themes through a process of coding and condensing 
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the codes, and finally representing the data in figures, table or a discussion." Thus, the 
categories and codes for Phase 2 analysis were not developed a priori, but were informed 
by the findings that emerged from the network cross-case analysis. The investigator 
coded the text of the mission, vision and value statements with descriptive labels, and 
then merged the groupings with similar labels to form broad 'first order' categories. This 
process was then repeated again to develop more specific 'second order' categories. 
From these, two aggregate dimensions emerged from the process to illuminate a 
distinction in organizational type of the key network actors. 

Study Validity and Limitations 
Validity 

This investigation is a comparative case study of five goal directed stakeholder 
networks, with the aim to indentify and explore patterns. Each stakeholder network is 
treated as a distinct case study. The threats to external validity of the study could be a 
concern given the small N of five organizational networks; however, no causal 
relationships or statistical correlations are being sought. The aim of this study to shed 
light on the research questions posed to help policy makers and program implementers 
better understand organizational networks through theory based empirical research. 

Given the mixed method, cross case design, there are low threats to maturation, 
testing effects, instrumentation and attrition. In an attempt to strengthen construct 
validity and consistency of measurement, the survey questions are adapted from Varda's 
(2012a) validated and field tested PARTER survey. However, with the focus on the 
organization, it is important to recognize that individual representatives completed the 
survey on behalf of the organizations. It is possible that these individuals might have 
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been unaware of all ties between their organization and the other organizations in the 
network (in the context of standardized developmental screening, referral and follow up 
for children in their community). It is also possible that some of the respondents' 
answers were influenced by their own personal biases or beliefs, and do not accurately 
reflect the quality of ties between their organization and others. This is inherent in all 
organizational network research. The respondents were the individuals actively involved 
in the network, representing their organization in the ongoing efforts to improve 
standardized developmental screening, referral and follow up for children in your 
community, so this threat is minimized to some extent. 

Although significant efforts were made to attain as high a response rate to the 
network survey as possible to improve the validity of the data, not every single 
organization completed the network survey. In the end, 61 organizations completed the 
survey out of a total of 81 organizations. This is an overall response rate of 75%, with 
individual network response rates ranging between 67%- 83%; however, it is important 
that the response rate within each network be as high possible to ensure that there is not 
an unequal representation of ties that could bias the results. 

There is a low threat to statistical regression as this research design does not select 
on extreme scores, does not have a second test, and the analysis is mathematically based 
instead of focusing on predictive relationships. Finally, while there is a medium threat to 
selection, because the study is based on one type of network: this study investigates 
stakeholder networks in only one specific subsystem of the health and public health care 
systems: that of early intervention (EI). This threat is partially overcome with the 
selection of five stakeholder networks. Further, the ultimate purpose is to identify 
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patterns that emerge, thereby shifting the focus of the findings to only those variables that 
were common across all sites, further reducing the threat to internal validity. Replicating 
this research with another set of organizational networks in a different context to explore 
whether the same trends hold true would help to mitigate the external threat to validity. 
Limitations 

It is important to recognize the limitations of any research. Of the four systems 
approaches detailed in Chapter I (systems organizing, system dynamics, system 
networks, and systems knowledge), this study is primarily focused on just one: system 
networks. The findings from the cross case analysis are thus limited to the relationships 
between the organizations involved in the stakeholder networks, and do not take into 
account all of the relationships within the EI system. 

Another limitation to the study stems from the inability to directly measure the 
increase in standardized screening, referral and follow through in each network 
community. The scope of such a performance measurement is well beyond the means of 
one investigator, and does not exist in secondary data sets at the level required for this 
investigation. Further, the ultimate aim of the networks is to enable children with 
developmental disabilities to reach their maximum development potential, an outcome 
that would be nearly impossible to quantify. This weakness was overcome to some 
degree by triangulating performance data from multiple sources: external stakeholders; 
members of the stakeholder networks, and Early Intervention Colorado. 

Further, data were collected at just one time from the networks. 
Interorganizational relationships are dynamic and always evolving. The findings would 
be strengthened with a research design that called for multiple points of data collection. 
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Finally, the inability to determine statistical correlation or causal relationships is 
another area of weakness for this investigation. Any generalizations from the study 
findings should be treated with caution. A larger N would help to overcome this 
weakness. 
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CHAPTER V 
FINDINGS 

This chapter shares findings of the cross-case analysis of five stakeholder 
networks operating within the early intervention system in Colorado, which spans the 
health and public health systems, along with systems of human services and education. 
The research questions that guide this study ask: Do community based stakeholder 
networks operating within the same context exhibit commonalities in network-level 
attributes and characteristics relative to their performance? Do the connectivity and 
structural positions of member organizations affect stakeholder network performance? 

To answer these questions, the first step was to analyze network performance and 
place the stakeholder networks on a performance continuum. The second step was to 
analyze each stakeholder network and compare all five cases to identify similarities or 
differences in demographics, structural and normative characteristics at the whole 
network level, including density, degree centralization and trust. 

The third step in the analysis sought to identify and analyze patterns of 
similarities and differences at the dyad and organizational levels of analysis, including 
degree centrality, relative connectivity, closeness centrality, betweenness centrality and 
brokerage (IVs), in the context of stakeholder network performance (DV). The fourth 
step in the analysis sought to explore and interpret the distinction in organizational type 
that emerged from the network cross case analysis findings, through content analysis of 
network members' mission and vision statements. 

Cross-case analysis was conducted utilizing UCINET and NetDraw (Borgatti et 
al, 2002) and the PARTNER Tool (Varda, 2008b) on the data set containing five 
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stakeholder networks (Nn=5); 81 total organizations (No= 81); 61 respondent 
organizations (Nr= 61); and 483 dyadic ties (Nt= 483). Content analysis was conducted 
utilizing NVivo 9 on the mission and vision statements of each organization in the five 
networks. Findings are presented in the sections that follow. 

To ensure confidentiality to the stakeholder network members, the networks have 
been coded A-E using the International Radiotelephony Spelling Alphabet: Alpha, Bravo, 
Charlie, Delta, Echo, in order of performance. 

Performance 

Both research questions require an understanding of network performance. Data 
on three separate performance measures detailed in Chapter IV were collected and 
analyzed to triangulate performance assessments for each of the stakeholder networks. 
Three distinct assessments were used to enhance construct validity and reliability of 
performance, the dependent variable in this study (Yin, 1994). The performance 
assessments for each stakeholder network are summarized below, and detailed in Table 
V.l. 
External Assessment 

The Assuring Better Child health Development (ABCD) Colorado leadership 
team assessed the performance of the stakeholder networks according to six key activities 
and deliverables to help inform community efforts and measure progress with their 
efforts to ensure standardized screening, referral and follow up for children. The six key 
activities and subsequent deliverables were described in Chapter IV and are attached in 
Appendix 2. Each stakeholder network was given an overall assessment based upon 
points earned for each of the six steps and deliverables. The raw assessments ranged from 
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a low of 22 to the highest score of 70, with a mean of 42 and a standard deviation of 18. 
The highest possible score was 100. To equally weight the three different performance 
assessments in the triangulated measure of network performance, each assessment is 
calculated as a percentage of highest possible points. 

This measure was also used to select the networks for inclusion in the study in a 
most-different cross case study design to deliberately compare cases that differ in their 
performance range (as assessed by ABCD Colorado) in order to identify patterns of 
attributes and characteristics relative to their overall performance. Upon case selection, 
two additional performance measures were identified to triangulate the performance 
measure for this study. These include: members' perceived performance in a self 
assessment and local EI data in an assessment of system output, as described in Chapter 
IV and summarized below. 
Members' Assessment 

Members of the stakeholder networks were requested, via survey, to self assess 
the network's efforts. The survey asked: "From your organization's point of view, how 
successful has this focus been for children in your community?" Respondents were 
asked to answer on a rating scale. The answers were weighted from 1= Not Successful to 
5= Completely Successful and averaged for a self assessment score. The self assessment 
scores ranged from an average of 3 to 3.5. The mean was 3.20, with a standard deviation 
of 0.23. To enable comparison across cases and ensure equal weight in the triangulated 
performance measure, the self assessment scores are calculated for each stakeholder 
network as a percentage of the highest possible score. For example, weighting the 
responses from 1 to 5, the total score for Alpha was 35. Ten organizations provided 
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responses, for a possible maximum score of 50. Thus Alpha's self assessment score was 
35/50= 70 percent. For ease of discussion, the decimal point is dropped and the 
percentage is equated to a score between 1 and 100, meaning that the score of 70 percent 
becomes 70. 
Local Early Intervention Assessment 

Part C of IDEA requires that the lead agency in each state monitor and supervise 
all early intervention services and programs within the state (EI Colorado, 2012). The 
degree to which each local community meets the EI targets where the stakeholder 
networks operates was calculated from the local EI data described in Chapter IV, which 
was the third measure used to triangulate network performance. Three of the five 
communities met or exceeded the target, earning a score of 100. The mean EI assessment 
was 92, with a standard deviation of 10. 

The three measures of performance are summed into a performance index. Based 
on this index, the mean performance of the five stakeholder networks in this study was 
199, with a standard deviation of 25. A summary of the network performance 
assessments are presented in Table V. 1 . 
Table V.l Network Performance. 



Stakeholder Network 


External 


Self 


Local EI 


Performance 




Assessment 


Assessment 


Assessment 


Index 


Alpha 


70 


70 


100 


240 


Bravo 


43 


62 


100 


205 


Charlie 


43 


68 


80 


190 


Delta 


22 


60 


100 


182 


Echo 


34 


60 


83 


177 


Mean 


42 


64 


92 


199 


SD 


18 


5 


10 


25 
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Stakeholder Network Alpha had the highest overall performance of the five 
stakeholder networks in this study, with a performance score of 240. The stakeholder 
network was assessed among the highest in the external assessment by the ABCD 
Colorado leadership team with an assessment of 70 out of a possible 100 (70%= score of 
70). The self assessment reflected a sense of accomplishment, with an assessment of 35 
out of a possible 50 (70%= score of 70). The distribution of responses from the self 
assessment is depicted in Appendix 6. The local early intervention data showed that the 
target had also been exceeded in Alpha's community, with 2.62 percent of children from 
birth to age three with Individualized Family Service Plans (IFSP), compared to the 
statewide target of 2.5 percent (exceeding 100%= score of 100). This indicates that 
children with developmental delays or disabilities are being identified early, and the 
referral and follow up processes between agencies are working to ensure that children are 
receiving early intervention services. The assessment also signified progress against the 
previous year's measure (EI Colorado, 2012). The stakeholder group has been working 
together towards standardized developmental screening, referral and follow up for 
children in their community for approximately 18 months at the time of the survey, and is 
among the smaller stakeholder networks in this study (4 l out of 5). 

Stakeholder Network Bravo had the second highest overall performance of the 
five stakeholder networks in this study, with a performance score of 205. The network's 
external assessment was 43 out of the highest possible of 100 (43%= score of 43). The 
self assessment reflected a sense of accomplishment, with an assessment of 56 out of a 
possible 90 (62%= score of 62). The distribution of responses from the self assessment is 
depicted in Appendix 6. The local early intervention data showed that the target had 
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been exceeded, with 3.16 percent of children from birth to age three with ISFPs than the 
statistically determined statewide target of 2.5 percent (exceeding 100%= score of 100). 
The assessment also signified progress against the previous year's measure (EI Colorado, 
2012). The stakeholder network has been working together for almost 6 years, and is the 
largest network in this study. 

Stakeholder Network Charlie had a performance score of 190. The network's 
external assessment was 43 out of the highest possible of 100 (43% = score of 43). The 
self assessment reflected a sense of accomplishment, with an assessment of 27 out of a 
possible 40 (68%= score of 68). The local early intervention data showed that the target 
had not yet been reached, with 1 .99 percent of children from birth to age three with 
ISFPs, which is lower than the statewide target of 2.5 (80%= score of 80). The 
assessment also reflected a lower percentage of children 0-3 with ISFPs than the previous 
year's measure (EI Colorado, 2012). The stakeholder network has been working together 
for approximately 3 years, and is among the smallest networks in this study. 

Stakeholder Network Delta had a performance score of 1 82. The network's 
external assessment was 22 out of the highest possible of 100 (22% = score of 22). The 
self assessment reflected a sense of accomplishment, with an assessment of 30 out of a 
possible 50 (60% = score of 60). The local early intervention data showed that the local 
target had been exceeded, with 2.77 percent of children from birth to age three with 
ISFPs, compared to the statewide target of 2.5 percent (exceeding 100%= score of 100). 
However, this assessment reflected a lower percentage than the previous year's measure 
(EI Colorado, 2012). The stakeholder network has been working together for 
approximately 3 years, and is mid-sized among the networks in this study. 
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Stakeholder Network Echo had the lowest performance of the five networks in 
this study, with a performance score of 177. The network's external assessment was 34 
out of the highest possible of 100 (34%= score of 34). The self assessment reflected a 
sense of accomplishment, with an assessment of 27 out of a possible 45 (60% = score of 
60). The local early intervention data showed that 2.07 percent of children from birth to 
age three with ISFPs, as compared to the statewide target of 2.50 percent (83%= score of 
83). This assessment reflected a lower percentage than the previous year's measure, 
however (EI Colorado, 2012). The stakeholder network has been working together for 
approximately 3 years, and is among the smallest of the networks in this study. 

The three performance measures for the five stakeholder networks are depicted in 
Figure V. 1 . 
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Figure V.l Comparison of Stakeholder Network Performance. 
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As Figure V.l illustrates, stakeholder network Alpha performed well, achieving 
the highest score in the external assessment. Stakeholder network Bravo tied with Alpha 
in the self assessment and local early intervention data, and scored second in overall 
performance. Stakeholder network Charlie was in the middle range of performance, 
scoring fairly consistently across all three measures. The stakeholder networks Delta and 
Echo are close to one another at the lower end of the performance assessments in this 
study, with one network scoring higher in the external assessment, tied in the self 
assessment and the other network scoring higher in the local EI data. 

All three of the measurements reflect an important element of network 
performance. The external assessment focuses on the processes and steps that are 
deemed necessary by ABCD and based upon the organizations' extensive experience 
about "standardized developmental screening and the referral and follow-up process that 
should occur once children are identified, as well as factors that need to be considered 
when multiple community entities are screening" (ABCD, 2012, p. 1). This measure also 
captures the networks' performance in improving the identification and referral stages of 
the EI system. This is weighted equally with the local early intervention assessment, 
which captures performance of the delivery of EI services. These two measures are 
weighted equally with the network members' own self assessment of network 
performance, a perspective that has also been identified as an important measure in the 
literature (Gray, 1995; Pro van & Milward, 1995) and focuses on the network's 
performance across the whole EI system. The stakeholder networks were then placed 
along a performance continuum based upon the triangulated performance measure (equal 
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weighting of external stakeholder assessments, network members' self assessments, and 
local EI assessments) as illustrated in Figure V. 2. 

Highest Performance Lowest Performance 




Figure V.2 Stakeholder Networks along a Performance Continuum. 

The figure reflects a continuum of performance for the five networks in this 
investigation only. 

Demographic Analysis 
Demographic Variables 

The first research question asks whether community based stakeholder networks 
operating within the same context exhibit commonalities in network-level attributes and 
characteristics relative to their performance. All five stakeholder networks were selected 
from the same relational of early intervention for the purposes of holding the network 
environment constant. All five networks aim to improve their local EI system by 
working towards ensuring standardized developmental screening, referral and follow up 
for children in their community. Although they formed to meet the unique needs of their 
community, they are have fairly similar levels of environmental stability, since EI is 
legislated at the federal level and managed at the state level, as described in Chapter II. 
Their resource munificence was also fairly stable, with a consistent level of technical 
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240 


12 


1.3 


1 


205 


27 


6 


4 


190 


10 


3 


5 


182 


18 


3 


2 


177 


14 


3 


3 


199 


16.2 


3.6 


N/A 


25 


6.7 


1.7 


N/A 



assistance, from ABCD. The demographics of the five stakeholder networks are 

summarized in Table V.2. 

Table V.2 Summary of Network Demographics. 

Stakeholder Network Performance Size Age County Size 

Rank 

Alpha 

Bravo 

Charlie 

Delta 

Echo 

Mean 
SD 

[Size: number or organizations in a network. Age: number of years the network has 
been operating in community. County size indicates the population of a county that the 
network operates in rank order relative to the others: small to largest. Mean scores and 
standard deviation are listed following the networks]. 

As indicated in Table V.2, the five stakeholder networks in this study range in size 
and age. The network sizes range from 10-27 organizations, with a mean of 16.2 and a 
standard deviation of 6.7. The median size of the five networks was 14 organizations. 
The age of the networks also ranged from 18 months to 6 years old, with a mean of 3.3 
and a standard deviation of 1 .7. The median age of the networks at the time of data 
collection was 3 years old. Patterns of size, age, and comparative county size are 
explored in the cross case analysis. 
Cross Case Analysis: Demographic Variables 

Beginning with the demographic variables, the cross case analysis examines each 
network's demographic variables and performance. This exploratory phase of the 
research investigates the first research question, as there are few theories that correlate 
whole network measures of stakeholder systems building networks to performance. 
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Figures V.3 through V.5 illustrate each network's level of performance relative to 
network size, network age and relative county size. 
Network Size 

Figure V.3 depicts the performance levels of the five networks by size. The 
performance level of each network in the study has been plotted along the y axis and the 
size of the networks has been plotted along the x axis. Each point represents the 
performance and size of a network. 
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Figure V.3 Variation in Network Performance by Size. 

As the graph illustrates, there was no clear pattern or evidence of a relationship in 

a network's level of performance relative to its size. The highest performing network is 
one of the smallest in the study, while the next highest performing network is the largest 
in the study. The peaks and valleys can be observed in the Order 4 polynomial trendline, 
which has an R-squared value of 1 . A trendline is most reliable when its R-squared value 
is at or near 1, and thus this is a perfect fit of the curve to the data. The trendline shows 
the variation in the performance of the networks relative to their size. This result is 
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important in public health, where collaboration is measured, in part by counting the 
number of organizations. This is finding is explored in Chapter VI, in the context of EI, 
health and public health systems. 
Network Age 

Figure V.4 depicts the performance levels of the five networks in the study by the 
number of years that the network has been working together. Each point represents the 
performance and age of a network. 



Network Performance by Age 
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Figure V.4 Variation in Network Performance by Age. 

As the graph depicts, there was no clear pattern or evidence of a relationship in a 

network's level of performance relative to its age. The youngest network scored lowest 
on the performance continuum, while the oldest network was next on the performance 
continuum. The peaks and valleys can be observed in the Order 3 polynomial trendline. 
The trendline has an R-squared value of 0.99, which is a good fit of the curve to the data. 
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Relative County Size (Population) 

The stakeholder networks in this study operate in five counties in Colorado, 
which were each ranked by population (1-5) relative to the others. Figure V.5 depicts 
each network's size along the x axis, with performance again on the y axis. Each point 
represents the performance the network and the relative population of the county in 
which it operates. 
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Figure V.5 Variation in Network Performance by Relative County Size. 

As the graph depicts, there was no clear pattern or evidence of a relationship in a 

network's performance relative to its county size. The highest performing network is in 
one of the smallest counties in the study, while second highest performing network 
operates within one of the largest counties. The peaks and valleys can be observed in the 
Order 3 polynomial trendline. The trendline has an R-squared value of 0.99, which is a 
good fit of the curve to the data. 

Overall, no clear patterns of performance existed in the cross case comparison of 
demographic variables. 
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Whole Network Cross Case Analysis 
Whole-Network Variables 

Based upon responses to the network surveys, sociomatrices were developed and 
characteristic and attributes at the whole network levels were calculated. Table V.3 
details different characteristics of the whole network that are of importance in addressing 
the first research question. 

Table V.3 Summary of Whole Network Characteristics. 

Stakeholder . Density Degree Centralization Trust 

Network Level Centralization Level 

Low 76.5% 

High 67.4% 

Mod Low 76.2% 

Mod High 78.7% 

Mod Low 76.3% 

75.0% 
4.4% 

[Density: percentage of ties present in relation to the total number of possible ties. 
Degree centralization: relationship between centrality of all members in network. The 
lower the centralization score, the more similar the members are in terms of their number 
of connections (e.g. decentralized). Trust: the percentage of how much members trust one 
another. Mean and standard deviation are in parenthesis]. 

To enable comparison, categorical levels were also determined for the structural 

network variables. A description of High was used to describe measures that were 

greater than the mean by more than one standard deviation; Moderately High was used to 

describe measures greater than the mean by less than one standard deviation; Moderately 

Low was used to describe measures less than the mean by less than one standard 

deviation; and Low was used to describe measures less than the mean by more than one 

standard deviation (Masseurs, 2012). 



Alpha 


81.8% 


High 


21.80 


Bravo 


33.4% 


Low 


55.40 


Charlie 


77.8% 


Mod High 


27.80 


Delta 


57.5% 


Mod Low 


47.80 


Echo 


60.4% 


Mod Low 


37.20 


Mean 


62.2% 




38.00 


SD 


19.3% 




13.84 



129 



Density 

The density of the stakeholder networks in this study ranged from 33.4-81.8 
percent, with a mean of 62.2 and a standard deviation 19.3. Since density is an index of 
the degree of connection between actors within the network, it can provide some insight 
about the connectedness of a network, but it is sensitive to the number of organizations. 
Figure V.6 depicts the performance levels of the five stakeholder networks by their 
density levels. 



Network Performance by Density 
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Figure V.6 Variation in Network Performance by Density. 

As the graph depicts, there was no clear pattern or evidence of a relationship in a 
network's level of performance relative to its density. Alpha was the most dense with a 
density level of 81.8% and Bravo was the least dense with a density of 33.4%. The peaks 
and valleys can be observed in the Order 3 polynomial trendline. The trendline has an R- 
squared value of 0.97, which is a good fit of the curve to the data. 
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While density has been interpreted as a measure of network integration 
(Masseurs, 2012), it only captures the number of ties present, not how they are organized 
within the network. To understand network integration, it is important to also investigate 
how the network structures are organized with respect to the centrality of its members, 
such as a network's degree of centralization. A whole network centrality measure is the 
degree of centralization, which is calculates the structural positions of all organizations in 
a network relative to each other. The measure indicates whether a network is centralized 
around a single organization, or group of organizations, or whether it is more 
decentralized. 
Degree Centralization 

The networks' degree of centralization ranges from 21.80-55.40. This means that 
none of the stakeholder networks in this study are centralized around a single 
organization. The cross case analysis of performance relative to degree centralization is 
illustrated in Figure V.7. 
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Figure V.7 Variation in Network Performance by Degree of Centralization. 
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As the graph depicts, there was no clear pattern or evidence of a relationship in a 
network's level of performance relative to its degree of centralization. The highest 
performing network, Alpha, is the least centralized network in the study, with a low 
degree of centralization of 21.8 percent, while the network next along the performance 
continuum, Bravo, had a high degree of centralization of 55.40 percent, relative to the 
study mean. The two lower performing stakeholder networks in the study, Delta and 
Echo had a moderate high and moderate low degree of centralization with 47.80 percent 
and 37.20 percent respectively, relative to the study mean. The peaks and valleys can be 
observed in the Order 2 polynomial trendline. The trendline has an R-squared value of 
0.95, which is a good fit of the curve to the data. The lack of a linear relationship in a 
network's level of performance relative to its degree centralization adds to the 
inconsistent (and contradictory) research findings on this measure. The implication of 
the finding will be explored in Chapter VI. 

This is a whole network aggregate measurement based on the centrality of 
organizations within the network. By itself, it cannot provide insight into the structural 
position of specific organizations within the networks, the focus of research question two. 
The results of centrality and betweenness centrality, which measure structural positions at 
the individual organizational level are investigated within the context of performance and 
presented later in the chapter. 
Overall Network Trust 

Based upon findings in the literature, it was expected that there would be higher 
levels of trust in the higher performing stakeholder networks. Each survey respondent in 
the five networks was asked to rate every other organization that they had a tie with in the 
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network on three dimensions of trust: reliability, support of mission and openness to 
discussion in a scale ranging from "Not at all" (coded 1); "A small amount" (coded 2); 
"A fair amount" (coded 3); and "A great deal" (coded 4) (Varda et al, 2008). These 
scores were then aggregated and divided by the highest possible rating to determine the 
percentage trust in the organizations. As summarized in Table V.3, ranges from 67.40- 
78.70 percent, with a mean of 75 percent and a standard deviation of only 4.4 percent. 
With 100 percent trust reflecting that all network members trust all other network 
members completely, the findings illustrate a key similarity among networks: all five 
stakeholder networks in the study have high levels of trust among organizations. 

Since all stakeholder networks in this study have high levels of trust, no pattern in 
a network's level of performance relative to its overall trust could be ascertained. The 
findings showed that the relatively lower performing stakeholder networks had very 
similar levels of trust than the higher performing networks in the study. Thus, the study 
findings indicate that while trust may be asserted in the literature as a necessary condition 
for network performance, it is not sufficient. The importance of these findings in EI, 
health and public health systems are discussed in Chapter VI. 
Network Graphs 

Structural characteristics of the whole network are also evident in network graphs, 
which visually represent the networks. Graphs illustrate a set of nodes, which represent 
the organization in the network, along with a set of ties that signify how they are 
connected. Figure V.8 illustrates ties in stakeholder network Alpha. 
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Figure V.8 Graph of Stakeholder Network Alpha. 

The network looks quite dense, with many organizations connected to many other 
organizations. This is supported by a density calculation of 81.8 percent, detailed 
previously. The network does not appear to be centered around any single organization 
and the number of ties that each organization has with other organization ranges from 4- 
1 1 . This observation is supported by its network degree of centralization of 21 .80 
percent, meaning that the network is decentralized. 

The ties in this graph are dichotomous (coded as 1 = present or 0= absent). The 
ties also have a parameter, for Figures V.8 above and V.9 that follows, each organization 
or program must have worked with another organization/program on issues related to 
standardized developmental screening, referral and follow up for children in their 
community at least once a month for the existence of a tie. Ties can also be valued with 
strength of relationship and type of relationship, as they are later in the chapter. 
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The organizations themselves also have attributes: in Figure V.8, each 
organization is coded by type (behavioral health; early childhood; education; public 
health; health care; or early intervention) and distinguished by color (red, pink, green 
yellow and peach). 

Graphs of all the stakeholder networks in this study are shown side by side for 
visual comparison in Figure V.9. 
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Figure V.9 Stakeholder Network Graphs. 
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Examination of the network graphs highlights patterns in whole network 
structural characteristics, such as the variables of size, density and degree centralization. 
In comparing the graphs in Figure V.9, the differences in network size and density are 
readily apparent. Stakeholder network Charlie is clearly the smallest, with the fewest 
number of organizations, while stakeholder network Bravo is the largest, with the most 
number of organizations. Comparing their basic structures, it becomes apparent that 
there is a range of density among the networks and they are all fairly decentralized, as 
discussed. 

Aside from the whole-network variables, the graphics reveal other findings about 
organizational positions and connectivity. The local Early Childhood Council (ECC), 
local public health department or agency (PH), and local Community Centered Board, the 
EI service provider contracted by the Colorado Department of Human Services (EI) are 
often near the center of the networks, with more ties than other organizations; while the 
local primary care providers, behavioral health center and early childhood preschools are 
located around the periphery of the networks. This indicates that the ECC, PH and EI 
organizations likely have higher degree centrality and closeness centrality than the rest of 
the network members in all five networks. 

In summary, the findings indicate that there are no trends between whole network 
characteristics and attributes and performance, and the stakeholder networks seem to be 
arranged similarly. It is thus important to investigate the attributes and characteristics as 
the organizational level, which is the focus of the second research question, to better 
understand why and how performance differs. The structural positions of specific 
organizations in Figure V.9 provide some clues to the organizations' roles within the 
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networks. Network theorists have also identified organizational level measures that 
indicate an organization's level of influence in the network, including betweenness 
centrality, relative connectivity and brokerage, which are investigated within the context 
of network performance in the next section. 

Cross Case Analysis- Organization Level 

Role and Influence of Public Health 

Regardless of overall network integration, there are often one or more actors that 
are in a position of greater influence in a network. Power and influence in a network are 
conceptualized and measured in the network literature with relative connectivity, 
closeness centrality, betweenness centrality, and brokerage measures. Table V.4 
summarizes these key measures for the local public health agency (PH) in each network. 
The measures were also analyzed for two other key network members: the local Early 
Childhood Councils (ECCs) and the local EI programs (Els). 
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Table V.4. Measures of Structural Position and Influence of Network Members. 



Organization 


Degree 


Relative 


Closeness 


Betweenness 


Brokerage, 


Type 


Centrality 


Connectivity 


Centrality 


Centrality 


Normalized 


Alpha 


PH 


10 


90% 


0.92 


0.03 


0.29 


ECC 


10 


100% 


0.92 


0.04 


0.29 


EI 


10 


75% 


0.92 


0.01 


0.24 


Mean 


9 


73% 


0.87 


0.02 


0.24 


SD 


2.13 


23% 


0.13 


0.02 


0.14 


Bravo 


PH 


16 


59% 


0.72 


0.04 


0.76 


ECC 


22 


100% 


0.87 


0.01 


0.78 


EI 


11 


60% 


0.63 


0.12 


0.57 


Mean 


9.04 


38% 


0.60 


0.02 


0.57 


SD 


5.59 


24% 


0.10 


0.04 


0.19 


Charlie 


PH 


9 


93% 


1 


0.04 


0.44 


ECC 


8 


100% 


0.90 


0.01 


0.32 


EI 


8 


82% 


0.90 


0.13 


0.45 


Mean 


7 


76% 


0.84 


0.05 


0.24 


SD 


1.89 


22% 


0.13 


0.04 


0.18 


Delta 


PH 


15 


84% 


0.89 


0.03 


0.69 


ECC 


8 


52% 


0.65 


0.00 


0.03 


EI 


15 


94% 


0.89 


0.01 


0.69 


Mean 


9.78 


58% 


0.73 


0.01 


0.29 


SD 


4.66 


26% 


0.14 


0.02 


0.29 


Echo 


PH 


7 


83% 


0.68 


0.00 


0.19 


ECC 


12 


83% 


0.93 


0.03 


0.56 


EI 


12 


100% 


0.93 


0.10 


0.63 


Mean 


7.86 


57% 


0.74 


0.02 


0.38 


SD 


2.88 


23% 


0.13 


0.03 


0.18 



[Mean and Standard Deviation reflect entire network. PH= Local public health department or 
agency; ECC= Local Early Childhood Council; EI= Local Community Centered Board, the EI 
service provider contracted by the Colorado Department of Human Services] 
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Degree Centrality 

Degree centrality counts the number of ties each organization has with other 
organizations in the network. The measure is relative to network size, and thus can only 
be compared within networks. To enable comparison, categorical levels were also 
determined for the variables. A level of High was attributed to measures above the mean 
by more than the one standard deviation; Moderately High was attributed to measures 
above the mean, but within one standard deviation; Moderately Low was attributed to 
measures below the mean, but within one standard deviation; and Low was attributed to 
measures below the mean by more than one standard deviation. The results indicate that 
the PH, ECC and EI organizations all had moderate-high or high degree centrality within 
their network, meaning that they had more ties than most organizations in their network. 
Relative Connectivity 

Building upon the degree centrality measure, relative connectivity measures both 
the number and quality of connections between organizations in each network. The 
measure is based upon: a) each organization's degree of centrality, b) the level of trust 
that other organizations have for them and c) how much they value their relationships 
with the other organizations. While relative connectivity scores ranged from 18-100% 
within the stakeholder networks, a striking pattern emerged in the findings. There was a 
strong pattern in a network's level of performance relative to the local early childhood 
council's relative connectivity, as illustrated in Figure V.10. 
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Network Performance by 
Relative Connectivity of ECCs 
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Figure V.10 Network Performance by Relative Connectivity of ECCs. 

As indicated by the chart and the polynomial trendline (R 2 = 0.53), the ECCs had 
the highest relative connectivity in the higher performing networks; while their relative 
connectivity was lower in the two lower performing networks. 

In addition, a pattern emerged indicating almost the opposite finding with the 
local early intervention programs, which had the highest or among the highest relative 
connectivity scores in the lower performing networks, as illustrated in Figure V. 1 1 . 
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Network Performance by 
EI Program's Relative Connectivity 
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Figure V.ll Network Performance by Relative Connectivity of Els. 

As indicated by the chart and polynomial trendline (R 2 = 0.59), the EI service 
providers have lower relative connectivity in the higher performing networks; had 
moderate relative connectivity in the network in the middle of the performance 
continuum, and high relative connectivity in the lower performing stakeholder networks. 

The data do not allow for analysis of a causal relationship or statistically 
significant correlation; however, the polymer trendline on both graphs supports the 
pattern across the five networks in this study. This importance and implication of this 
finding is discussed in Chapter VI. 
Closeness and Betweenness Centrality 

The closeness centrality measure can be used to identify those organizations with 
the shortest paths to all of the other organizations in the network, which provides 
important insight into an organization's location in the network, relative to all of the other 
organizations. The measure is an inverse, with the shortest distance = 1 and the longest 
distance = 0. 



142 



The variation among organizations scores and among networks varied so that 
there was no pattern in a network's level of performance relative to the closeness 
centrality of its highly connected members. The local public health agency (PH) in each 
network had moderate-high or high levels of closeness centrality (shortest distance) in 
four of stakeholder networks (Alpha, Bravo, Charlie, and Delta). In stakeholder network 
Echo, the lowest performing network, PH had a moderate-low level of closeness 
centrality (longer distance). The early childhood council in each network had high levels 
of closeness centrality (shortest distance) in all but stakeholder network Alpha, where it 
had a moderate-high level, and Delta, where it had a moderate low level (longer 
distance). The early intervention agency had moderate-high levels in the three higher 
performing networks and high levels (shortest distance) in the lower two performing 
stakeholder networks. Both public health and early childhood councils had the same or 
higher closeness centrality than the early intervention agency in the top three performing 
stakeholder networks, while the local early intervention program had the same or higher 
closeness centrality in the two lower performing networks. 

Betweenness centrality measures varied tremendously between the local public 
health agency, early childhood council and early intervention agency across the networks, 
but at least two of the three organizations had high and moderately high centrality in each 
network. This finding indicates that two of the three key actors in the stakeholder 
networks were in a position between otherwise unconnected organizations in the network. 
This position has a degree of power and can also become a point of failure for the 
network. These implications will be discussed in Chapter VI. There was no linear 
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pattern in a network's level of performance relative to the betweenness centrality of its 

highly connected members. 

Brokerage 

Brokerage is supported in the literature as a good measure for influence in a 
network. A broker is a structural measurement that indicates the degree to which an 
organization spans a structural hole in the network structure. There was a strong pattern 
in a network's level of performance relative to the local early intervention service 
provider position as a broker, as illustrated in Figure V.12. 
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Figure V.12 Network Performance by Brokerage of Els. 

As indicated by chart and the polynomial trendline (R 2 = 0.84), EI programs were 
in relatively strong brokerage positions in the lower performing stakeholder networks; 
were in brokerage positions of more moderate strength the networks in the middle of the 
performance continuum; and less in a brokerage position in the higher performing 
networks. 
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In summary, the cross-case analysis revealed that ECCs had the highest relative 
connectivity in the three highest performing stakeholder networks in the study. This 
means that in each high performing network, the ECCs not only had among the highest 
number of connections, these connections were also of a high quality: the relationships 
were with organizations that trusted the ECCs and with organizations that the ECCs 
valued. The local public health agencies (PH) also showed a fairly similar pattern, with 
higher relative connectivity in the higher performing networks in this study and lower 
relative connectivity in the lower performing networks in this study, as summarized in 
table V.5. 
Table V.5. ECC and PH Relative Connectivity. 



Stakeholder Network Relative Connectivity Relative Connectivity 
ECC PH 



Alpha 


100% 


90% 


Bravo 


100% 


59% 


Charlie 


100% 


93% 


Delta 


52% 


84% 


Echo 


83% 


83% 



The positive association between ECCs / PH's high relative connectivity and 
network performance should be taken with some caution given the R-squared values of 
0.53 and 0.59. However, the trend is worthy of further investigation through content 
analysis presented later in the chapter. 

Conversely, the cross case analysis revealed a negative association in the EI 
program's relative connectivity, which was highest in lower performing networks and 
lower in high performing networks. In fact, in the lowest performing stakeholder 
network, the EI had the highest relative connectivity. A similar such pattern was found 
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regarding the EI s brokerage roles as well, which was higher in lower performing 

networks and lower in high performing networks. 

Table V.6. EI Program's Relative Connectivity and Brokerage. 



Organization 


Relative 


Brokerage, 


Type 


Connectivity 


Normalized 




EI 


EI 


Alpha 


75% 


0.24 


Bravo 


60% 


0.57 


Charlie 


82% 


0.45 


Delta 


94% 


0.69 


Echo 


100 % 


0.63 



The negative association with the El's relative connectivity and brokerage roles is 
unexpected, and is explored in the content analysis later in this chapter, and discussed in 
Chapter VI. 

Building upon theories of network research detailed in Chapter IV, the attributes 
of primary care providers are investigated in the following section. 
Integration of Primary Care Providers 

The integration of primary care providers is measured by the degree to which they 
have strong ties with the other key actors in the system (public health and early 
intervention), based upon activity type. A stronger tie is indicated when survey 
respondents were to classify the kinds of activities that their organization' relationship 
with every other organization in the network entailed (in the context of standardized 
developmental screening, referral and follow up for children in their community)? The 
respondents could select from the following classifications: 
• "None; 
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• Cooperative Activities: such as exchanging information, closing the loop 
on referrals, attending meetings together, and offering resources to 
partners; 

• Coordinated Activities: Include cooperative activities in addition to 
intentional efforts to enhance each other's capacity for the mutual benefit 
ofprograms; or 

• Integrated Activities: In addition to cooperative and coordinated activities, 
this is the act of using commonalities to create a unified center of 
knowledge and programming that supports work in related content areas." 

A stronger tie is indicated when organization engages in coordinated activities, which 
would include the implementation of standardized screening in well-child checkups and a 
referral and follow through process with the local early intervention agency, and 
integrated activities. 

By means of example, Figure V.13 depicts a graph of stakeholder network Alpha, 
where ties represent coordinated activities. Different from the network graphs that were 
presented earlier in Figure V.9, the overall network connectedness is not the focus here; 
rather, it is the ties between primary care providers (represented as brown nodes) and the 
local public health agency (PH, represented as a yellow node) and the local early 
intervention program (EI, represented as a peach node). 
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Primary Care 3 




Primary Care 2 
Group Key 

Local behavioral health center (BH) \ / Primary Care f 

£ Early Childhood (ECC/EL) 

£ Local school district/BOCE (ED) 

Q Local public health agency (PH) 

A Local primary care provider (PC) 

Q Local EI service provider (EI) 

A Parent/Family 

A Other nonprofit 

Figure V.13 Graph of Stakeholder Network Alpha, Coordinated Activities. 

In Figure V.13, the PH agency has ties with all three primary care providers in the 

direction of the primary care provider, indicating that it works to enhance the primary 

care provider's ability to undertake standardized screening and referral. An arrow 

represents the direction of the relationship as indicated by the survey respondent. The 

direction towards the primary care providers is important, as the ties reflect a relationship 

that goes beyond simply exchanging information, closing the loop on referrals, attending 

meetings together and offering resources to partners. The directed tie means that the PH 

agency is intentional about its efforts to enhance the primary care provider's capacity for 
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the mutual benefit of programs. The ECC also has ties with all three primary care 
providers, some ties are directed both ways, but all are directed towards the primary care 
providers. The EI in stakeholder network Alpha also has directed ties with two of three 
primary care providers. Figure V.14 depicts graphs of coordinated ties for the five 
stakeholder networks for comparison. 
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Figure V.14 Stakeholder Networks, Coordinated Activities. 
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Upon comparison of the network graphs illustrating coordinated activities, it is 
clear that primary care providers and other service providers including the local mental 
health center and early childhood providers (preschools) are largely on the periphery in 
all networks. This is a similarity the stakeholder networks share, regardless of 
performance. However, a strong pattern emerges in the number and direction of ties 
between primary care providers and the key actors in the network (PH, ECC and EI) and 
the overall network performance. 

Comparing the ties across stakeholder networks, in the context of performance, a 
striking pattern becomes clear: the primary care providers have more ties that reflect 
coordinated activities with public health, early intervention and early childhood councils 
in the higher performing networks and fewer and fewer ties reflecting such activities in 
the lower performing stakeholder networks. In stakeholder network Bravo, the ECC has 
directed ties with five of the ten primary care providers. The EI has one directed tie with 
a primary care practice. In stakeholder network Charlie, PH has a directed tie with the 
one primary care provider in the network; however neither ECC, nor EI have a tie. In 
stakeholder network Delta, the PH has one directed tie to one of the four primary care 
providers in the network, and the EI and EEC have no ties with any of the primary care 
providers. Finally, in stakeholder network Echo, the PH, EI nor ECC have a tie with the 
primary care providers in the network. These findings have important implications, 
given the segmentation of the EI system and the call for primary care provider to 
undertake standardized screening. 
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Content Analysis 

Organizational Perspectives: Systems Building vs. Service Provision 

The study research design allowed for further investigation into the preliminary findings 
from the network analysis to learn more about the 'key' organizations in the stakeholder 
networks, indicated by high relative connectivity and high brokerage, and see if there is a 
relationship with performance. The most striking finding was the clear distinction of the 
Early Childhood Council (ECC) as being a key actor in all high performing networks, 
with a pattern in a network's level of performance relative to the ECC's structural 
position and role and their levels of connectivity and brokerage in the network. These 
same results emerge for PH agencies, but not to the same extent. Further, there were a 
number of inverse patterns in a network's level of performance relative to the 
Community Centered Board's (EI )'s structural position and role and their levels of 
connectivity and influence in the networks. 

In order to better understand and interpret these findings, content analysis was 
conducted to illuminate the similarities and differences between these organizations. All 
mission, vision and value statements that were publically available on the Internet were 
coded and analyzed in NVivo by first order categories, meaning that the statements were 
clustered into common themes. The same process was then repeated to code the data into 
second order themes to ultimately categorize the organizations into two aggregate 
dimensions: those with a systems building focus and those with an individual service 
provider focus. TableV.7 presents the results. 
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Table V.7 Classifications of Member Organizations. 



System Focus Individual Focus of Service Providers 

Local public health agencies (PH) Local early intervention program (El) 

Local Early Childhood Councils (ECC) Local school district or Board of Cooperative 

Education (ED) 

Local pediatrician or primary care provider (PC) 
Local early learning centers/preschool (EL) 
Local behavioral health center (BH) 



Two clear dimensions emerged from the qualitative coding process: a systems 
approach, in which systems building or system level collaboration were explicitly stated 
as a core mission or vision for the organization, and a service provider approach that 
focused first and foremost on providing quality services at the individual level. 
Organizations with a system focus emphasized system change in their mission and vision 
statements as part of what the organization aimed to achieve: they purposefully worked 
toward system level goals. By contrast, the organizations with an individual service 
provider focus aimed to provide high quality services to best meet the needs of their 
clients/patients. Their vision focused on individual care and service. 

For example, local public health agencies emphasized achieving their mission 
through collaboration. One PH's mission statement is one sentence long, ending with an 
emphasis on collaboration: "through people, prevention and partnerships ." Another PH's 
vision statement starts: "Through partnerships and collaboration [we] will lead our 
community..." From the language in their mission and vision statements, it was clear that 
it established the importance of interorganizational relations in its endeavors to serve 
their residents. 

Local Early Childhood Councils (ECCs) emphasize their aim to improve the 

systems that serve young children through system level collaboration. One ECCs 
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mission stated that: "champions for children and families. .[by] serv[ing] as a vehicle that 
brings together agencies'" . . . "on a seamless system of care for young children." The 
mission and vision statements of the ECC emphasize that "collaboration is essential" and 
that the councils are "rooted in the values of community, collaboration and leadership on 
behalf of children from birth through age eight." 

The system focus differed from the individual focus of service providers who are 
committed first to their clients and patients. Their focus was on "providing the highest 
quality of service" to their clients and "high quality care" to their patients. For example, 
one primary care provider described its purpose as providing "Evidence Driven, Patient 
Centered, High Quality, Friendly Health Care." Overall, the primary care organizations' 
vision and mission statements rarely referred to collaboration and partnership, and when 
they did, it was framed as a method to better serve the clients. For example, teamwork 
was listed as a core value for many health care providers, but they regard it as teamwork 
between physician and patient to better serve the patient. 

The local early intervention programs shared commonalities with both 
dimensions; however, they were coded as having the individual focus because their 
commitment, first and foremost, was serving their clients. Their interest in collaboration 
and working with the community was consistently framed as a method to better serve 
their clients and expand their clients' opportunities; it was regarded as a means to an end. 
The mission and vision and value statements referred to dual goals centered on 
"enriching lives" of clients and "strengthening communities. " Many statements 
referenced being a "vital partner in our community"; working "in partnership with 
families and communities to expand opportunities' '; and "build[ing] reciprocal 
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relationships with community partners that aim to increase opportunity and high quality 
services." Systems building or systems change was not listed as a service or area of 
emphasis for any of the local early intervention providers. 

The dimensions, and subsequent classification, of system focused organizations 
and service provider that emerged from the content analysis was considered in the 
context of the network analysis findings described previously. Given the varied 
perspectives and priorities of the organizations involved in each stakeholder network, the 
application revealed that there were patterns in a network's level of performance relative 
to the a structural position and influential role of the system builders in the networks. 
The higher performing networks had at least one system focused organization that was 
well connected and trusted, in the powerful and influential position to broker 
relationships and facilitate flow within the network. By contrast, the lower performing 
networks had a service provider in this position. The findings do not indicate that any 
one organization is any more capable than other; it is the perspective and emphasis that 
they bring to the network in that position. The findings suggest that the perspective of a 
key organization in the stakeholder network, measured by high relative connectivity, is 
related to performance. 

Conclusion 

In summary, the findings indicated that there were no relationships in whole- 
network characteristics relative to network performance. It was also shown that trust, 
which might be necessary to network performance, as indicated in the literature, was not 
sufficient for performance. 
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The stakeholder networks tended to be organized around the local Early 
Childhood Councils (ECCs), local public health agency (PHs) and local early 
intervention program (Els). Two of the three organizations served as connectors within 
each stakeholder network. While both public health and early intervention are federally 
legislated agencies in the EI system, and were thus expected to be central actors in the 
networks, the Early Childhood Councils' central role in the networks was an unexpected 
finding. The positive relationship between the ECCs' relative connectivity and network 
performance was indicated in a clear pattern in the cross case analysis. Unexpectedly, 
networks with EI programs in a broker position had poorer performance. These 
contrasting patterns were further investigated through content analysis. The distinction 
between the types of organizational perspective emerged from analysis of organization 
mission, vision and value statement, which helped to illuminate the importance of an 
organization's focus to network performance. The findings indicated a positive 
relationship between network structures with a systems-focused organization in a key 
position and overall network performance; while findings indicated a negative 
relationship between network structures with a service provider in a key position and 
overall network performance. Finally, strong relationships between the three central 
actors (ECC, PH and EI) and primary care providers, as measured by coordinated 
activities to efforts to enhance mutual benefit, was found to be positively related to 
network performance. 
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CHAPTER VI 

DISCUSSION 

Linking Network Attributes and Characteristics to Performance 

This chapter discusses the research findings presented in the preceding chapter. A 
brief summary of the study is offered; the findings are then interpreted with regard to the 
literature and discussed in the context of the study research questions. Conclusions for 
health and public health systems are then offered, as are recommendations for the 
application of study findings and future research. 
Summary of Investigation 

The reliance upon interorganizational relationships in health and public health 
systems is extensive; however, there is a shortage of evidence that focuses on stakeholder 
network performance. Responding to this gap and the call for a systems approach, the 
study's first research question asked whether community based stakeholder networks 
operating within the same context exhibit commonalities in network-level attributes and 
characteristics relative to their performance. The second question investigated 
organizational level variables, asking if the relative connectivity and structural positions 
of member organizations affect stakeholder network performance. 

In a mixed method, cross case research design, the characteristics and attributes of 
five stakeholder networks were analyzed and compared. Whole network, dyad and 
organizational measures were examined through social network analysis (SNA) and 
content analysis, and analyzed with performance measures based on stakeholder 
interviews, self assessments and local early intervention assessments. Focusing on the 
dynamic interactions of public health agencies with public and private organizations that 
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affect health, the five stakeholder groups were goal-directed networks, working to ensure 
standardized developmental screening, referral and follow up for children in their 
Colorado communities. The stakeholder networks were mobilized to overcome the 
segmented system of early intervention services. 

The study found that there were no patterns in the cross case analysis of network 
level variables relative to performance; while linear patterns were found in organizational 
level variables relative to performance. The findings showed that high performing 
networks had system-building organizations in positions with high relative connectivity, 
while the lower performing networks had a service provider in those positions. Primary 
care providers also had strong ties with the key actors in high performing stakeholder 
networks, and these ties gradually decreased in number when compared with networks 
lower along the performance continuum. Results also showed that trust is necessary, but 
not a sufficient condition of network performance. 

Exploring Research Question One 

This study first asked: 

Do community based stakeholder networks operating within the same 
context exhibit commonalities in network-level attributes and characteristics 

relative to their performance? 

This research falls into an emerging area of network research that focuses on the 

relationship between network structures at the whole network level, such as density or 

network degree centralization, and overall network performance, again measured at the 

whole network level (see Provan & Milward, 1995, Rosenheck et al, 1998; Provan et al, 

2007). 
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Network Structure 

While a model of network effectiveness (Pro van & Milward, 1995) exists in the 
literature that correlates whole network measures of integration with overall network 
effectiveness, subsequent research that focused on the validation of this work has found 
contradictory findings (Rosenheck et al, 1998). Further, low density/high centrality 
networks are typical of innovation networks, but the relationships between network 
integration and performance have not be empirically tested (Dhanaraj & Parhke, 2006). 
This area of organizational network research is still fragmented and under developed, and 
although there is considerable interest, there is "no research available to inform practice 
about the "right amount" of integration in a network" (Provan & Lemaire, 2012, p. 653.) 
Thus, without a clear model in the literature, the first research question sought to explore 
structural integration. Specifically, the research contributes to the literature by analyzing 
the relationships between network integration and performance. The research design 
enabled a unique comparison of five locally formed stakeholder networks operating in 
their local system of early intervention that all shared the same goals and fairly stable 
levels of resource to explore commonalities and patterns. 

Network integration was conceptualized by: network density, which provides 
insight into the overall basic connectedness of an organization; and the degree of 
centralization, which illuminates to the degree to which the network is structured around 
certain organizations relative to others. Utilizing the most-different cross case design, the 
findings included considerable range in both measurements, although none of the 
networks were highly centralized. 
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The findings indicated that community based stakeholder networks operating 
within the same context exhibit do not exhibit commonalities in network-level attributes 
and characteristics relative to their performance. The findings revealed no patterns in 
network demographics or whole network variables relative to performance. Further, 
there was no relationship between a network's level of integration, conceptualized with 
whole-network measurements of density and degree centralization, and its performance. 
The findings are counter to Proven and Milward's (1995) model of network 
effectiveness; however, that model has not yet been validated by other researchers. 
Further, Proven and Milward based their model upon research focused on a service 
delivery network, which is a different type than the stakeholder networks in this 
investigation. 

The absence of a pattern is significant for health and public health systems 
research, where policy makers and public health leaders are looking for guidance around 
the most effective and efficient way to leverage limited resources through organizational 
networks. Although a topic of great interest in network literature, there is no clear 
answer about what type of organizational network structure is most effective. The 
findings in this study suggest that whole network integration does not drive performance. 
While the highest performing network had a high density and low degree of 
centralization, another high performing stakeholder network had a low density and 
comparatively high degree of centralization. Given the similarities in the networks' 
purpose and formation, it is surprising that they varied so much in whole-network 
integration, and that there was no pattern in relation to their performance. Perhaps the 
reason that a "right" level of network integration has yet to be formulated in 
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organizational network research is that there simply is no "one size fits all" solution. 
Consideration of just the structure and integration of organizational networks may be too 
simplistic of a perspective to account for the complexities within the dynamic social 
structures of organizational networks. The study findings indicate that organizational 
level attributes and characteristics may be what really matters to network performance. 
Network Trust 

Network and collaboration research literature emphasize the importance of trust 
for effective interorganizational relationships and networks (Uzzi, 1996, 1997; Bryson et 
al, 2006; Thomson et al, 2008b). Further, trust is a "common theme in virtually all 
definitions of interorganizational networks" (Provan et al, 2007, p. 643), with empirical 
evidence that overall trust is an integral condition for network performance (Uzzi, 1996; 
Provan et al, 2003; Zaheer et al, 1998; Huxham & Vangen, 2005; Ring & Van de Ven, 
1994; Bryson et al, 2006; Thomson et al, 2008b). Based upon these findings, one would 
expect to see a pattern of higher trust in the higher performing stakeholder networks, and 
a lower trust in the lower performing stakeholder networks. 

Trust and performance were analyzed for each stakeholder network at the whole 
network level to identify if such a pattern existed in the cross case analysis. Study results 
showed that all five stakeholder networks in this investigation had fairly high levels of 
trust, ranging from 67.40- 78.70 percent (where 100 percent trust would mean that all 
members of the network trust every member at the highest level) (Varda, 2012). Further, 
there was no pattern in performance relative to trust: trust levels were high, regardless of 
performance. 
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The study findings do not contradict the literature's assertion that trust is an 
integral condition for performance; however, the findings reveal that trust is not a 
sufficient condition for network performance. The most-different cross case design of 
this study enabled disconfirming research, finding stakeholder networks with high trust 
levels, but lower performance. Since trust levels were reported high in all the stakeholder 
networks, including those with poorer performance, the findings suggest that either trust 
is not a key driver of performance, or it may be over-estimated by survey respondents. 

This result has significant application in health and public health systems, where 
network leaders and members often spend considerable time building trust within their 
organizational networks or collaboratives. It is important for practitioners to understand 
that those efforts, while vitally important, will not be enough to develop and manage high 
performing stakeholder networks. 

In summary, the findings in response to research question one indicate: 1) no 

whole-network relationship to performance; and 2) trust levels were counter to 

expectation and are an insufficient condition for network performance. These findings 

suggest that something else explains network performance. Deeper analysis of the dyads 

between and among organizations within the networks is the focus of the second research 

question. 

Exploring Research Question Two 

Do the relative connectivity and structural position of member 
organizations affect stakeholder network performance? 

This research focuses on the impact of an organization's relational and structural 

attributes and characteristics, measured at the individual member and dyad level, as 

opposed to the whole network level. Seeking to better understand correlations between 
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organizational level variables and overall network outcomes, this second question focuses 
the investigation on the impact of the structural positions of certain types of 
organizations, how they are connected, and the roles they play in the network. 
Organization Positions and Roles 

As detailed in Chapters I and II, the fundamental charge of public health is 
"fulfilling society's interest in assuring conditions in which people can be healthy" (IOM, 
1988, p. 140). Further, the Social Security Act, Title V Maternal and Child Health 
Program legislates that public health agencies implement family-centered, community- 
based, systems of coordinated care for children with special healthcare needs (MCHB, 
2012a). Since local public health agencies offer few direct EI services, they must 
coordinate, connect and collaborate with primary care, human services and education to 
ensure effective identification, screening and delivery of services for children with 
developmental delays. This is especially important given that health and public health 
literature identify challenges with the segmented system of early intervention. Thus it 
would be expected that public health agencies (PHs) would be in a central and highly 
connected position in high performing networks, and that they would take on a brokering 
role in the stakeholder networks, working to connect organizations that make up the 
disconnected parts of the system. 

Counter to expectation, the study findings revealed that public health agencies had 
a fairly central role in most networks; however, they were not necessarily the most 
connected or in the strongest brokerage positions. 

A striking finding of the investigation was the emergence of the early childhood 
councils (ECCs) in these roles. As discussed in Chapter V, the ECCs had the highest 
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relative connectivity in the three highest performing stakeholder networks in the study. 
This means that in each high performing network, the ECCs not only had among the 
highest number of connections, these connections were also of a high quality: the 
relationships were with organizations that trusted the ECCs and with organizations that 
the ECCs valued. The positive association between their high relative connectivity and 
the network performance thus illuminated the ECCs integral role in the stakeholder 
networks. 

This finding builds upon the organizational network literature, where research 
correlates brokerage roles with an organization's influence within a network (Hanneman 
& Riddle, 2005). In the higher performing stakeholder networks, the ECCs were all well 
connected and used these connections to broker new relationships between previously 
unconnected parts of the networks. In doing so, they built influence within the network 
(Hanneman & Riddle, 2005). While no 'right' level of whole network integration can be 
found in the organizational network literature, in the context of these networks, it appears 
that there may be one 'right' organization to take on an integral and influential role in the 
networks to achieve higher performance. With such high centrality and trust for the 
ECCs, they may be the ideal broker organization, with the ability to connect 
organizations and to pass along important information. 

The ECCs aim to "improve and sustain the availability, accessibility, capacity and 
quality of early childhood services for children and families throughout the state." 
(Colorado Department of Education [CDE], 2013, para.l) This systems building focus 
may enable them to more effectively and efficiently bring together and work with 
partners. Systems building efforts are rarely funded or measured in the public sector, and 
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organizations often balance so many directives that they need to prioritize funded and 
measured responsibilities first. ECCs are in the unique position of being able to focus 
entirely on systems level efforts, which is aligned with their core mission, funding and 
evaluation. This may contribute to their success as an influential connector in the high 
performing stakeholder networks. 

The fact that ECCs were in these roles more than public health agencies 
highlights two opportunities for health and public health leaders: 

a) The findings provide an opportunity to further consider the role of ECCs in 
local early intervention systems since they were an integral actor in these 
stakeholder networks, in a system in which they may not always be 
considered a primary stakeholder by policymakers. 

b) There is an opportunity for public health agencies to better leverage their 
multiple community connections and systems perspective to further improve 
the EI system in their communities by assuming a more active connector 
and/or broker role. 

Conversely, the local early intervention program (EI) was in an almost inverse 
position to the ECCs, relative to network performance. The cross case analysis revealed 
a negative association in the El's relative connectivity, which was highest in lower 
performing networks and lower in high performing networks. In fact, in the lowest 
performing stakeholder network, the EI had the highest relative connectivity, meaning 
that they had the highest number of quality connections with other organizations in the 
network. 
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A similar pattern was found regarding the Els' brokerage roles as well, which was 
higher in lower performing networks and lower in high performing networks. Kilduff 
and Tsai (2003) found that the role of a broker is only advisable for organizations that 
carry legitimacy in the context of the network's efforts. The negative association with 
the Els' brokerage role is surprising and counter to expectation, given the legitimacy that 
the EI service provider would bring to a network focused on EI. Fernandez and Gould 
(1994) found that in order to be an effective broker in networks in the policy arena, actors 
needed to be perceived as relatively impartial to the policy initiatives. Although the 
networks in this study are goal directed stakeholder network rather than policy networks, 
a similar perception may be influencing EI providers' ability to be successful brokers. 
Other network actors may perceive El's role as a broker to be a veiled effort to enhance 
Els' service provision, whereas they may perceive ECC's role as a broker to unite actors 
to improve the segmented system for the benefit of children. Both organizations are 
working toward the same goal, but the primary focus of their organization may shape the 
way their message and efforts are received by the other organizations in the networks. 

The stark contrast of: the positive relationship between network structures with 
ECC in a key position and network performance; and the negative relationship between 
network structures with EI in a key position and network performance, indicated that 
something about the differences between those two organizations was diving network 
performance. Content analysis identified the primary distinction between the 
organizations from their perspective. Both the ECCs and PHs have a system-building 
focus as a core part of their missions and organizational values. The Els, on the other 
hand, while committed to community engagement and partnerships to better serve their 

166 



clients, first and foremost focus on providing quality services on an individual basis. All 
the other stakeholder network members were services providers, also focused first on 
serving their clients. 

This distinction is significant in light of the cross case network analysis findings: 
all high performing stakeholder networks had an organization with a systems building 
perspective in an influential and central role, as evidenced by high relative connectivity 
and brokerage measures; whereas the lower performing stakeholder networks had a 
service provider in an influential and central position. In fact, it was an inverse pattern. 
Contributions to the Literature 

An important conclusion of this study is that one cannot consider the whole 
network without examining the structure of the component parts. These findings provide 
important insight into organizational network literature, as empirical research that 
investigates the performance of whole networks "is still lacking" (Lemaire & Provan, 
2009, p .2; Provan et al, 2007; Provan & Kenis, 2008). This area of study is 
fundamentally about understanding the impact of network attributes, characteristics and 
conditions on network outcomes, and although the research is quite rich in describing 
various positions in a network, little attention has been paid to the type of organization in 
such a role, and its impact on network performance. This study provides insight not just 
in relation to various member organizations' impact on the network, but to whether the 
organization maintains a systems building or service provision emphasis in the network. 

Another important finding in this study is the strong distinction between systems 
focused organizations and service provider organizations as key connectors and brokers 
in the networks, and their impact on network performance. This is an area that we still do 
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not fully understand and requires further research. It may be that the EI organizations' 
service focus impacts the perception of the other organizations in the network, and affects 
the other organizations' willingness to stay involved and committed. Recall that EI needs 
the rest of the system to effectively function so that children with developmental delays 
are identified early and referred to them for services. EI providers may be perceived as 
telling everyone what needs to be done so that they can deliver services, whereas the 
systems focused organization may be perceived as wanting improve the systems for 
everyone's benefit, so that they can all better serve children. 

A third consideration is that the service providers simply do not have the skills or 
capacity to serve as effective network connectors or brokers. Service providers, by 
definition, are focused first and foremost on providing quality services to their clients 
and/or patients. They are often compensated only through direct service provision and 
may not have the time, resources, expertise, skills, or even interest, in playing an 
important connecting role in the networks. This is another under-investigated area in the 
organizational network literature (Conklin, Lusk, Harris, & Stolee, 2013). Conklin and 
colleagues recently set out to define and describe the role of brokers in a knowledge 
exchange network and found quite simply that "you need people with skills to enable 
groups to come together" (Conklin et al, 2013, p. 5). Their findings indicated that most 
knowledge brokers described their primary role as connecting actors with other actors, 
information, ideas, and educational resources; and their secondary role as coordinating 
actors. Other network actors identified brokers as important in promoting mutual 
understanding among network actors (Conklin et al, 2013). Along the same lines, Miles 
and Snow (1986) found that an effective network broker is a 'strategy maker' that 
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coordinates and facilitates organizations within networks, but does not command. By 
their very nature, the systems-focused organizations often take on a coordination and 
facilitation role in their day-to-day systems-building work. Further investigation is 
required to understand whether the systems-focused organizations are thus better 
equipped to take on such roles in networks than the service providers. 

The goal congruence of a network broker and the overall network should also be 
considered in light of the study findings. The stakeholder networks in this study aim to 
ensure standardized developmental screening, referral and follow up for children in their 
community. Thus, the networks are working to improve the system that leads up to the 
provision of services, along with the connection to the EI service provider. As identified 
in the content analysis, the EI providers' organizational efforts are focused on providing 
quality EI services to the children who are referred to them. Although EI providers are 
an integral stakeholder in the EI system, their goals primarily focus on a different part of 
the system (the service delivery) than the networks' goals (the identification, referral and 
follow-though). Lundin (2007) found that goal congruence affects cooperation among 
organizations, and specifically, that organizations must have both trust and goal 
congruence for effective joint action. Although EI providers' high trust scores 
demonstrate that the organizations are perceived as trustworthy by other network 
members, the misalignment of their goals in relation to networks' goals may be inhibiting 
their ability to be an effective connector and broker. ECCs, on the other hand, aim to 
improve the systems that serve young children through system level collaboration. The 
ECCs' goal is broader than the networks' goal, but the networks' goal aligns perfectly 
with the ECCs' goal. Also a highly trusted member of each network, the ECCs have 
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both trust and goal congruence, which may enable them successful connect and broker 
key relationships in the stakeholder networks to ultimately enable the networks to 
perform better. The affect of goal congruence on network roles is not yet well 
understood in the literature. 

Finally, the notion of accountability of the network members is also worthy of 
further consideration in light of the findings. The EI providers in Colorado are 
accountable to the Colorado Department of Human Services (CDHS), who are 
accountable to Congress, for the delivery of high quality and timely EI services to 
children with developmental delays. Thus, it may not be appropriate to expect them to 
spend their efforts and resources to be effective connectors or brokers in stakeholder 
networks. Although the networks aim to improve the EI system through enhanced 
screening, referral and follow up for children, the legislated requirements of the EI 
providers focuses only on the provision of services (to those referred). Thus, Els' 
involvement in these networks is not strictly part of their legislative charge. Of course, 
the networks aim to overcome the segmentation in the system and improve the outcomes 
for children with developmental delays, and so their involvement is necessary for the 
networks to succeed, but their participation as a connector or broker may be stifled by the 
fact that they are not accountable to these networks, but rather to CDHS and Congress. 
On the other hand, ECCs and PH agencies are accountable for systems level change, and 
thus the performance of the network is congruent with their accountability. 

The findings also have significant application for health and public health 
systems, where a key focus is on getting the right organizations to come to the table and 
be a part of an interorganizational effort. The findings illustrate that it is not simply a 
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matter of having the right organizations engaged in the network, they must also be in the 
right positions . The networks in the study, which are working to ensure standardized 
developmental screening, referral and follow up for children in their communities, are 
goal-directed stakeholder networks with an aim to improve the local segmentation and 
implementation of the EI system. For such networks, the study results show that having 
an organization with a systems-building perspective in key and influential position is 
crucial for network performance. These findings help to define what a systems focused 
organizations is, in very tangible way. 
Strength of Ties 

Given the segmentation in the EI system, and the call for integration of primary 
care providers, one would expect to see primary care providers well connected in the high 
performing networks. Network research tells us that being well connected does not mean 
that an organization needs to have ties with all members of a network. It is important to 
be strategic in the number, type and direction of ties, but also in the strength of ties. 
Karckahrdt (1992, 2010) argues the benefits of strong ties, while Granovetter (1973, 
1992) found that there are benefits in weak ties. Further, representatives of organizations 
have a limited amount of time and energy available for maintaining ties with other 
organizations, especially strong ties, which typically require more of an investment of 
time and energy. It is generally much easier to maintain a high number of weak ties 
(Pro van & Lemaire, 2012). Building upon the literature, it was expected that primary 
care providers in high performing networks would have stronger relationships with public 
health and early intervention than with other organizations in the network. Given the 
system context and networks' goals, it would be important that the relationships not 
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simply indicate an awareness of the organizations or frequency of contact; rather the ties 
should represent coordinated activities to ensure system effectiveness. Thus a strong 
relationship was conceptualized as a relationship that included cooperative activities, in 
addition to intentional efforts to enhance each other's capacity for the mutual benefit of 
programs. 

Comparing the relationships to and from primary care providers with other 
organizations in each stakeholder network, a striking pattern became clear: the findings 
revealed a pattern in the number of strong relationships between primary care providers 
and public health, early childhood councils and early intervention. Specifically, the PHs, 
ECCs, and to some degree, Els, had more relationships that reflected coordinated 
activities with primary care providers in the higher performing networks and fewer 
relationships that reflected such activities in the lower performing stakeholder networks. 
The positive association of the primary care providers' strong ties with key actors and the 
networks' performance helps to illuminate which relationships are worthy of investment 
to foster and maintain as strong relationships, and where weak relationships are 
sufficient. The direction of these strong ties towards the primary care providers is also 
important, indicating that the ECCs, PHs and EI s were working to enhance the primary 
care provider's ability to undertake standardized screening and referral, and follow 
through. The findings support P3's assertion that the level of integration of the primary 
care providers within an early intervention stakeholder network impacts the network's 
contribution to public health strategies at the local levels. 

The findings build upon the network literature to add to the fields' understanding 
of the role of strong and weak ties in networks. Uzzi (1997) found that strong ties enable 
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increased problem solving capacity; Hansen (1999) found that they can provide richer 
and more effective transfer of information; while Karckhardt (2010) found that strong ties 
can be trusted sources of influence. However, organizations are limited in the number of 
strong ties they can maintain, and strategic decisions about with whom to have strong ties 
is important. Research has brought tremendous insight into both strong and weak ties, 
but a greater understanding of an effective mix of strong and weak ties is also critical. 
The findings show that key relationships mattered in this study: strong relationships 
between disconnected parts of the EI system were important for network performance, 
and ultimately EI system performance. There was no pattern relative to performance for 
PC providers' other relationships. Since it is only possible to maintain a limited number 
of strong collaborative relationships, it is integral that the network's goals and the broader 
system context are considered. 

These findings have significant implications for health and public health systems, 
given the segmentation of the EI system and the call for primary care providers to 
undertake standardized screening. The study results indicate that strong ties with ECCs, 
PHs and Els may help primary care providers to implement such screening tools to better 
identify children with developmental disabilities, thereby helping the network to achieve 
its goals and strengthening the local EI system. While the health literature indicates that 
up to half of all children with developmental delays are not identified until they enter 
school (Mackrides & Ryherd, 201 1), there is a consensus in the field that pediatric 
physicians have both the opportunity and expertise to identify children in need of EI 
services, and are thus the ideal candidates for helping to improve the segmented system 
(AAP, 2001). These findings are also in line with the recent and significant attention 
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focused on medical homes as a mechanism to help ensure quality in health care for 
children with disabilities, developmental delay or other special health care needs. The 
medical home model requires that children receive family-centered care from a physician 
or nurse who can also provide ready access to referrals and care coordination when 
needed (Strickland et al, 201 1). 

Many pediatricians and family medicine physicians do not regularly use 
standardized screening tools, even though there is evidence that a screening tool will 
more effectively and accurately identify children who may have a developmental delay 
(HHS, 2000; Rydz et al, 2005). Thus the directionality of these ties is also paramount 
given the need to integrate primary care physicians into the EI system for the important 
role of identification, screening and referral. The findings show that the ECCs, PHs, and 
to some degree, the Els, were intentional about their efforts to enhance the primary care 
provider's capacity for the mutual benefit of programs. The findings reinforce the 
importance of thinking strategically not only about the types of relationships, but also 
about the purpose and cost and benefits of the relationships in a network aimed at 
improving health and public health outcomes. 

Conclusions for Early Intervention, Health and Public Health Systems 
Interorganizational relationships are inherent in the health and public health 
systems where public, private and nonprofit organizations work together to ensure that 
America becomes "a society in which all people live long, healthy lives" (HHS, 2012, 
Message from the Secretary, para. 1). It is thus so important that policy leaders and 
practitioners are armed with the knowledge to prompt strategic and thoughtful discussion 
to foster stronger inclusion, involvement, and commitment where it most makes sense 
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and will have the greatest impact. This study aimed to provide insight into stakeholder 
network performance in the context of early intervention, which crosses the health, public 
health systems, along with education and human services systems. More specifically, the 
study aimed to develop evidence about patterns in a network's attributes and 
characteristics relative to its performance, adding to a much needed knowledge base to 
guide the use, formation and management of networks. Refraining the study findings to 
that end, these three conclusions are offered for consideration. 
Conclusion 1: Perspectives of Key Actors Matters 

The diverse group of public and private organizations and other stakeholders 
detailed in this dissertation, work together in multiple settings with overlapping health 
related goals and missions, ultimately trying to contribute to the health of society (IOM, 
2012a). For goal based networks, it is integral not only to align the network membership 
with the goals the network aims to achieve, but to align the perspectives of key actors 
with the network goals. Specifically, for networks aiming to improve health and public 
health systems, a systems level focus of key actors may be critical for network 
performance. In this study of goal directed, community based stakeholder networks; key 
actors in the network that had a systems focus were positively associated with network 
performance, while key actors that were service providers were not as well suited for 
these key positions or role, and were negatively associated with network performance. 
Conclusion 2: Integration of Primary Care can Strengthen Outcomes 

As with any research of interorganizational relationships, it is important to 
consider the context of the relationships under investigation. As detailed in Chapter II, 
stakeholder networks were mobilized to overcome the segmented system of identification 
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and delivery for early intervention (EI) services by, in part, helping to bridge a gap 
between primary care with public health, education, and human services. Recall that state 
and local agencies from public health, human services and education all must work 
together to implement federal policy to ensure a system of delivery for early intervention 
services. However, these agencies do not typically identify children for EI services. A 
systems approach highlighted the fact that the identification and referral to EI was a gap 
in the system because it was not addressed by the legislation. Further, without well 
defined inter-agency and inter-system processes, there are opportunities for children to be 
lost between systems. 

For these reasons, primary care providers have being called upon to fill this gap, 
and thus it is vital that they are integrated in the stakeholder networks. A considerable 
challenge they face is demonstrated in network and collaboration research: that it is 
unfeasible and unsustainable to maintain numerous strong ties (Krackhardt, 2010). This 
would especially be the case for physicians, as their primary focus is on providing 
individual care. This is further supported by Varda et al.'s (2008) findings that it was not 
only difficult for public health leaders to maintain interactions with collaborating 
organizations, but likely unsustainable. Furthermore, PCs are service, not system, 
oriented. Taken together, it would be unfeasible to expect that primary care providers 
could build and sustain many strong relationships in these stakeholder networks. 
However, it is critical that they are engaged to be able to bridge the segmentation that 
exists in EI and enable the network to leverage its collective strengths and achieve its 
goals. 
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Strategically focusing on how to connect primary care providers, and which 
relationships are worth the investment of time and effort to build and maintain is critical, 
so that relationship budgeting can make the most out of the primary care providers' 
involvement. This study revealed that when PHs and ECCs, and to some degree, EI s 
,worked to enhance primary care providers' capacity to implement screenings, the 
network performed well, and the entire system benefited. 
Conclusion 3: Bigger Is Not Necessarily Better 

Rather than simply building a large network, Burt (1995) has argued that the 
pattern of ties is more important than the size of one's network. The study findings 
support this notion, as there was no pattern in a network's level of performance relative to 
its size. This is significant in health and public health systems where organizational 
networks and efforts are measured in part by simply counting members. As briefly 
discussed in Chapter I, the National Public Health Performance Standards Program 
(NPHPSP) assesses PHs by the degree to which they are: 

• "Identifying potential stakeholders who contribute to or benefit from 
public health, and increase their awareness of the value of public health; 

• Building coalitions to draw upon the full range of potential human and 
material resources to improve community health; and 

• Convening and facilitating partnerships among groups and associations 
(including those not typically considered to be health-related) in 
understanding defined health improvement projects, including preventive, 
screening, rehabilitation, and support programs" (CDC, 2012, NPHPSP, 
Local Instruments: Local Public Health System Performance Assessment). 
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These assessments provide no guidance or incentives for being strategic about the 
relationships between and among organizations in a stakeholder network. 

Instead of expecting the PH to connect and interact with every organization that 
might possibly support overall population health in some way, organizational network 
research can be used to distinguish and facilitate connections, as detailed in this chapter, 
so that strong ties are established and fostered where they can have greatest impact, and 
weak ties are maintained so they can provide access to new information. For example, PH 
agencies may be most efficient serving as a broker in systems building network efforts, 
but may be most efficient as a peripheral member of a service delivery network. 
Organizational networks in health and public health hold the promise of enabling more 
efficient and effective use of resources through strategic leveraging, economies of scale 
and collective action (Bryson et al, 2006); however, further research is needed in this 
area to guide strategies to enable networks to be as effective and efficient as possible. 

Future Research Needs 

Interorganizational relationships and networks are a large part of the public health 
system infrastructure, and a "high priority uncertainty" (Consortium, 2012, S72). The 
more evidence that can be produced about how the characteristics and attributes of 
partnerships affect their performance, the more public health strategies and services can 
be organized, financed and delivered to leverage the strengths of these interorganizational 
relationships, while minimizing their weaknesses. This investigation focused on applied 
research with the aim to contribute insight to early intervention, health and public health 
leaders and practitioners by investigating how different attributes and characteristics of 
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the whole network and the relationships between organizations within the network are 
linked to network performance. 

Specifically, the research contributes to the literature by assessing the relationship 
between different characteristics and attributes of stakeholder networks and network 
performance with the conclusion that one cannot consider the whole network without 
examining the structure of its component parts. The strength of this study's findings for 
health and public health systems comes from the mixed-method cross-case analysis of 
five similar, but locally formed goal-directed stakeholder networks. The findings revealed 
no whole network relationship to performance, while they indicated a positive 
relationship between network structures with a systems-focused organization in a key 
position and overall network performance; and a negative relationship between network 
structures with a service provider in a key position and overall network performance. 
Based upon these results, it appears that there may be one 'right' organization to take on 
an integral or influential role for improved network performance. There is much more to 
learn about this notion, and this would be a fascinating new research direction. This study 
could be replicated with a greater number of cases or different types of networks to 
investigate whether the patterns hold. One could use an entirely different research design 
to investigate where there are one or more types of organizations that can most 
effectively lead organizational networks, and why. 

The distinction between service providers and organizations with a systems focus 
emerged from the network and content analysis. The need for systems level change in 
public health is reinforced in the literature, yet system-building efforts are rarely funded 
or measured in the public sector. Public management literature demonstrates that 
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managers prioritize those tasks that are measured and that have funding attached. A new 
approach to better understanding the implications of systems level funding could 
investigate the funding trail of federal, state and private sector funds dedicated toward 
change at the system level, or focus on the accountably of organizations within systems 
building efforts. In this study, ECCs are in the unique position of being able to focus 
entirely on systems level efforts, which is aligned with their core mission, funding and 
evaluation. This may contribute to their success as key actors in the high performing 
stakeholder networks. The fact that EI programs are funded for the individual services 
provided, as in primary care, may have contributed to their lack of success as key actors 
in the stakeholder network. Furthermore, these findings raise new questions about why 
EI providers were not effective connectors or brokers in the stakeholder networks. Was it 
the way they were perceived by the network members because of their service delivery 
focus; the misalignment of their goals with the network's overall goal; or was something 
else altogether? This is an area that is not yet fully understood, but the study findings 
certainly show that something is at play here, and requires further research. The study 
findings also reveal that there may be a competency that network brokers require in order 
to be effective. That begs the question: is a certain skill set needed to broker or govern 
organizational networks? This area is underdeveloped in the network literature and 
would be a fascinating area of future research. 

Finally, this study found that trust, while consistency linked with performance in 
the collaboration and network literature, was not a sufficient condition for network 
performance. This finding brings on many questions about what else is needed, in 
addition to trust, to be sufficient for performance. Building upon that, would such a 
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condition be strong enough to replace, or supersede, the importance of trust? One could 
also investigate if there was a point of diminishing return on trust? Also, how would a 
lack of trust affect these stakeholder groups? Is trust between organizations difficult for 
the employees or representatives of an organization to critically assess? Is it over 
estimated in organizational network research? These kinds of questions have yet to be 
answered, but could inform the way organizations work together to most effectively and 
efficiently achieve outcomes that are not possible by working alone. 
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APPENDIX 



1. Early Intervention Process in Colorado 



'Identification of potentially eligible- child from primary referral source^ 
Primary referral sources Include, but are not limited to the follovtfng; 



Child Care Provider 
Concerned Individual 
Family Member 
HCP/CRCSM/Public Health 



Health Care Provider 
Hospital Personnel 
Primary Care Physician 
Social Service Agency 





A child should be referred due to any of the following situations: 

A) Score on a standardized developmental screen falls below the tool's 

empirical cutoff in one or more of the five domains; 

B) a diagnosis of an established condition (physical or mental) with a high 
probability pf resulting in significant developmental delays, {view the 
established condition database within the r&ferfaf section of 
www. eicoforadOrOrg),: 

C) significant parental and/or clinician concerns related to the child's 
development and/or behavior; or ^A 

D) child has risk factors for social emotional problems. 



Referral Source 



Early Intervention 
Colorado Program 



Fax referral form to Early Intervention 
Colorado Program at Community Centered 
Board (www.eicoloradcorg) 



Referral Source and 
Primary Care Physician 
informed of findings and 

recommendations for 
ongoing monitoring 

(with parent consent) 




Not Eligible 



n gible 



Initial Individualized Family Service Plan (IF$P) 

developed with family within 45 days of referral. In 
addition to service coordination, there are fourteen 
possible early Intervention services, such as: 

A) Developmental Intervention 

B) Occupational, Physical or Speech Language Therapy 
C} Social Emotional Intervention 




El Evaluation Team can 

refer to other appropriate 

community services 



SOURCE: Early Intervention Colorado, 2012. 
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APPENDIX 
2. Survey Invitation 



Your organization has been identified by the Assuring Better Child health Development 
(ABCD) State team as a community stakeholder working to ensure standardized 
developmental screening, referral and follow up for children your community. Thank 
you for all your work towards this important goal. Given your involvement in this work, 
your organization is being asked to be in a research study titled: Partners for Healthy 
Development. 

Study Purpose 

This study aims to learn more about the social structure of interorganizational 
partnerships in public health, specifically asking how different characteristics and 
attributes of partnerships affect performance. This study focuses on the relationships 
between organizations working together towards the same goal: working to ensure 
standardized developmental screening, referral and follow up for children in their 
community. A total of six stakeholder groups will be in this study, including all of the 
organizations that are a part of each stakeholder group. 

Partnerships and collaboration are inherent in the public health system where public, 
private and nonprofit organizations work together to assure the conditions for population 
health. This study aims to arm you with the knowledge to prompt strategic and 
thoughtful discussion to foster stronger inclusion, involvement and commitment where it 
most makes sense and will have the greatest impact to ensure standardized developmental 
screening, referral and follow up for children your community. More broadly, this study 
aims to provide insight into the dimensions of collaboration, adding to a much needed 
knowledge base to guide the use, formation and management of partnerships and 
collaboration for a strong public health system. 

Your Participation 

You are being asked to complete an online survey. This survey will include 20 questions, 
and will take about 20 minutes. It will list all the organizations identified by ABCD as 
working to ensure standardized developmental screening, referral and follow up for 
children in your community, and request that you indicate your involvement and 
relationship with the other organizations on the list. Taking part in this study is voluntary. 
If you choose to take part, you have the right to skip questions or stop at any time. 

Confidentiality and Anonymity 

No personal information will be collected. All questions are strictly about organizations 
(not individuals). This means that as a respondent to the survey, you are asked to 
represent your organization. The data will only ever be presented as aggregate scores 
(combined average of all survey respondents within your community) and at the network 
level (mathematical calculations about the stakeholder group as a whole). Analysis of the 
data will be presented as part of a research dissertation in written and oral form, but will 
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never single out and identify one organization's relationship with another organization. 
(This will always remain confidential). Aggregate findings will also be presented to you 
and other organizations in your community that have taken the survey, and to the 
Colorado ABCD State Team. The research data and notes will remain confidential. (All 
survey results, analysis documents and research notes will be password protected and 
kept on an encrypted computer or locked filing cabinet). 

Funding for the Study 

This study is part of a dissertation for a PhD in Public Affairs at the University of 
Colorado Denver School of Public Affairs by Robyn Mobbs. The National Coordinating 
Center for Public Health Systems and Services Research, with funding from the Robert 
Wood Johnson Foundation, has awarded Robyn the Assuring the Future of Public Health 
Systems Research: Dissertation Award to support this research. 

Contact Details 

The researcher leading this study is Robyn Mobbs. If you have questions or concerns, 
you may call Robyn at 303-204-0640. You can also call the Colorado Multiple 
Institutional Review Board (COMIRB) at (303) 724-1055. 
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APPENDIX 

3. Survey Questions 



Partners for Healthy Development: 

A network analysis of community stakeholders working to ensure 

standardized developmental screening, referral and follow up 

for children in six Colorado communities. 

1 : Please select your organization/program from the list <local organizations will be 
listed by name for each of the following categories (more than one organization per 
category is OK>: 

Behavioral health 
Child care provider 
Child Find/ BOCES 
Health Department 
Head Start 
Health care provider 
Home visitation 
Early Childhood Council 
Early Intervention 

Family Resource Center 

Parent/family member 

Religious Organization 

Service Provider 

2: How is your organization/program participating to ensure standardized developmental 
screening, referral and follow up for children in your community? 

3: How long have you been participating to ensure standardized developmental screening, 
referral and follow up for children in your community (in months)? 

4: Please indicate what your organization/program, or can potentially contribute, to 
standardized developmental screening, referral and follow up for children in your 
community. (Please choose all that apply). 

Funding 

In kind resources. If you are providing in kind resources. Please define. 

Paid staff 

Volunteers / volunteer staff 

Data resources, including data sets, collection and analysis 

Information/feedback 

Specific health expertise 

Expertise other than health 

Community connections 
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• Fiscal management 

• Facilitation/leadership 



5: What is your organization's most important contribution to standardized 
developmental screening, referral and follow up for children in your community? 

Funding 

In kind resources. If you are providing in kind resources. Please define. 

Paid staff 

Volunteers / volunteer staff 

Data resources, including data sets, collection and analysis 

Information/feedback 

Specific health expertise 

Expertise other than health 

Community connections 

Fiscal management 

Facilitation/leadership 

6: Please indicate which of the following outcomes have been achieved as a result of the 
focus on standardized developmental screening, referral and follow up for children in 
your community. (Please choose all that apply). 

• Clearer understanding of the developmental and behavioral needs of children 

• Clearer understanding of what is needed to better meet those needs 

• Promotion and formation of working relationships between health care practices 
and state and local programs, services, and resources, including Community 
Centered Board (by name), Child Find/BOCES 

• Increase in the number of health care practices that have implemented 
standardized developmental screening and referral 

• Promotion of early identification and referral 

• Improved referral follow through process 

• Increase in the number of referrals 

• Increased resource sharing 

• Increased knowledge sharing 

• Stronger link with local Community Center Board <listed by name> 

7: From your organizations' point of view, which of the above describes the most 
successful outcome of the focus on standardized developmental screening, referral and 
follow up for children in your community? (Please choose one). 

• Clearer understanding of the developmental and behavioral needs of children 

• Clearer understanding of what is needed to better meet those needs 

• Promotion and formation of working relationships between health care practices 
and state and local programs, services, and resources, including local Community 
Centered Board (by name), Child Find/BOCES 

• Increase in the number of health care practices that have implemented 
standardized developmental screening and referral 
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• Promotion of early identification and referral 

• Improved referral follow through process 

• Increase in the number of referrals 

• Increased resource sharing 

• Increased knowledge sharing 

• Stronger link with local Community Center Board <listed by name> 

8: From your organizations' point of view, how successful has this focus been for 
children in your community? 

• Not Successful 

• Somewhat Successful 

• Successful 

• Very Successful 

• Completely Successful 

9: What aspects of collaboration contribute to this success? (Please choose all that apply). 

• Bringing together diverse stakeholders 

• Meeting regularly 

• Exchanging information/knowledge 

• Sharing resources 

• Formation of collaborative relationships 

• Collective decision-making 

• Emphasis on personal relationships 

• Shared Funding 

• Shared Expertise 

• Other 

Network Questions: 

10: From the list, please select the organizations/programs with which you have an 
established relationship (either formal or informal). In subsequent questions you will be 
asked about your relationships with these organizations/programs in the context of 
standardized developmental screening, referral and follow up for children in your 
community <local organizations will be listed by name from the following sectors>: 

Behavioral health 

Child care provider 

Child Find/ BOCES 

Health Department 

Head Start 

Health care provider 

Home visitation 

Early Childhood Council 

Early Intervention 
Family Resource Center 
Parent/family member 
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• Religious Organization 

• Service Provider 

1 1 : At this point in time, how frequently does your organization/program work with this 
organization/program on issues related to standardized developmental screening, referral 
and follow up for children in your community? 

• Never/We only interact on issues unrelated to standardized developmental 
screening, referral and follow up. 

• Once a year or less 

• Every few months 

• Every month 

• Every few weeks 

• Every week 

12: What kinds of activities does your organization/program's relationship with this 
organization/program entail (in the context of standardized developmental screening, 
referral and follow up for children in your community)? 

• None 

• Cooperative Activities: such as exchanging information, closing the loop on 
referrals, attending meetings together, and offering resources to partners. 

• Coordinated Activities: Include cooperative activities in addition to intentional 
efforts to enhance each other's capacity for the mutual benefit of programs. 
(Example: Separate granting programs utilizing shared administrative processes 
and forms for application review and selection.) 

• Integrated Activities: In addition to cooperative and coordinated activities, this is 
the act of using commonalities to create a unified center of knowledge and 
programming that supports work in related content areas. (Example: Developing 
and utilizing shared priorities for funding effective prevention strategies. Funding 
pools may be combined.) 

13: How valuable is this organization/program's power and influence (in the context of 
standardized developmental screening, referral and follow up for children in your 
community)? *Power /Influence: The organization/program holds a prominent position 
in the community be being powerful, having influence, success as a change agent, and 
showing leadership. 

• Not at all 

• A small amount 

• A fair amount 

• A great deal 

14: How valuable is this organization/program's level of involvement (in the context of 
standardized developmental screening, referral and follow up for children in your 
community)? *Level of Involvement: The organization/program is strongly committed 
and active in the partnership and gets things done. 

• Not at all 
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• A small amount 

• A fair amount 

• A great deal 

15: How valuable is this organization/program's resource contribution (in the context of 
standardized developmental screening, referral and follow up for children in your 
community)? Contributing Resources: The organization/program brings resources to 
the partnership like funding, information, or other resources. 

• Not at all 

• A small amount 

• A fair amount 

• A great deal 

16: How reliable is the organization/program (in the context of standardized 
developmental screening, referral and follow up for children in your community)? 
^Reliable: this organization/program is reliable in terms of following through on 
commitments. 

• Not at all 

• A small amount 

• A fair amount 

• A great deal 

17: To what extent does the organization/program share a common vision of standardized 
developmental screening, referral and follow up for children in your community? 
^Mission Congruence: this organization/program shares a common vision of the end 
goal of what working together should accomplish. 

• Not at all 

• A small amount 

• A fair amount 

• A great deal 

18: How open to discussion is the organization/program (in the context of standardized 
developmental screening, referral and follow up for children in your community)? 
*Open to Discussion: this organization/program is willing to engage in frank, open and 
civil discussion (especially when disagreement exists). The organization/program is 
willing to consider a variety of viewpoints and talk together (rather than at each other). 
You are able to communicate with this organization/program in an open, trusting 
manner. 

• Not at all 

• A small amount 

• A fair amount 

• A great deal 
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19: To the best of your knowledge, how would your organization define success of this 
focus on standardized developmental screening, referral and follow up for children in 
your community? 

20: To the best of your knowledge, how would your organization describe the benefits of 
participating in this effort? 
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APPENDIX 



4. Stakeholder Network Self Assessment Responses 



Alpha Self Assessment 
Responses 



Completely Successful 

Very Successful 

Successful 

Somewhat Successful 

Not Successful 







figure A.l Self Assessment for Stakeholder Network Alpha. 



Bravo Self Assessment 
Responses 



Completely Successful 

Very Successful 

Successful 

Somewhat Successful 

Not Successful 







FigureA.2 Self Assessment for Stakeholder Network Bravo. 



Charlie Self Assessment 
Responses 



Completely Successful 

Very Successful 

Successful 

Somewhat Successful 

Not Successful 







figure A.3 Self Assessment for Stakeholder Network Charlie. 
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Delta Self Assessment 
Responses 



Completely Successful 

Very Successful 

Successful 

Somewhat Successful 

Not Successful 



figure A.4 Self Assessment for Stakeholder Network Delta. 



Echo Self Assessment 
Responses 



Completely Successful 

Successful 

Not Successful 



12 3 4 5 



figure A.5 Self Assessment for Stakeholder Network Echo. 
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